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The  objective  of  this  study  is  to  develop  a hybrid  segmentation  approach  used  in 
the  computer  diagnostic  system  for  malignant  melanoma.  The  approach  has  three  stages: 
(1)  PEDBOR;  (2)  LVDSO  quantization;  (3)  mapping. 

In  Stage  I,  a new  algorithm,  called  Pixel-Feature-Controlling  Edge  Detection 
Based  On  Regularization  (PEDBOR),  is  used  to  remove  noise  and  detect  edges 
simultaneously.  It  is  based  on  the  regularization  model,  but  has  several  critical 
improvements.  The  improvements  are:  (1)  simple  optimization  method,  which  allows 
employing  additional  knowledge  about  the  physical  nature  of  edges  to  control  the 
regularization  procedure;  (2)  a data  driven  threshold  function,  which  avoids  the  difficulty 
of  setting  a threshold;  (3)  classification  of  pixels  based  on  both  magnitude  and  orientation 
of  the  image  gradient;  and  (4)  new  line  process  functions  set  for  different  types  of  pixels 
based  on  desired  line  process  function  properties,  which  makes  the  whole  regularization 


VI 


adaptively  controlled  by  pixel  features.  These  empower  the  PEDBOR  algorithm  to  operate 
without  setting  parameters  and  result  in  a well-posed  and  well-conditioned  process  that  is 
robust  to  noise.  By  using  PEDBOR  in  Stage  I,  both  noise  removed  images  and  edge 
images  with  high  quality  can  be  obtained.  They  provide  good  references  for  Stage  II  and 
Stage  III. 

In  Stage  II,  a quantization  method  using  iterative  largest-variance-direction- split- 
optimization  (LVDSO)  in  the  histogram  domain  of  the  color  space  is  applied  to  the  noise 
removed  image.  It  is  fast.  In  the  quantization,  an  oversegmenting  parameter  is  used  to 
avoid  the  difficulty  of  setting  an  exact  parameter  and  to  resolve  the  problem  of 
inconsistencies. 

Stage  III  is  a mapping  procedure.  It  uses  the  edge  image  fi*om  Stage  I as  the 
reference  of  true  regions,  and  uses  the  segmented  image  from  Stage  II  as  the  reference  of 
true  edges  to  remove  false  edges  and  false  regions. 

Experimental  results  indicate  that  the  final  segmentation  images  have  high  quality, 
that  is,  they  are  uniform,  compact,  dissimilar  and  consistent,  and  have  accurate  and 
continuous  boundaries. 


CHAPTER  1 
INTRODUCTION 

1.1  Motivation 

Malignant  melanoma  is  a common  and  deadly  dermatologic  disease.  Based  on 
statistics  released  by  the  American  Cancer  Society  in  1991  [Fri  91],  malignant  melanoma 
is  the  eighth  most  frequently  diagnosed  cancer  (See  Fig.  1-1).  In  that  year,  there  were 
30,000  new  cases  of  malignant  melanoma  and  6500  deaths  from  the  disease.  Figs.  1-2  and 
1 -3  give  two  statistics.  The  first  one  shows  the  lifetime  risk  for  an  individual  American  in 
different  years.  From  this  we  can  see  that  currently  among  105  new  babies,  one  will 
develop  malignant  melanoma.  The  second  figure  shows  the  incidence  rate  and  death  rate 
of  malignant  melanoma  in  different  years.  The  curve  tells  us  that  the  incidence  rate  is 
increasing  exponentially  and  the  death  rate  is  climbing  linearly.  Although  the  death  rate  has 
also  continued  to  increase,  survival  percentage  has  doubled  from  40%  in  1940s  to  80%  in 
1980s.  The  increase  in  survival  is  entirely  attributed  to  early  diagnosis.  Therefore  an  early 
and  correct  diagnosis  is  very  important.  To  establish  an  early  diagnosis,  many  visual 
features  of  malignant  melanoma,  such  as  color,  size,  border,  shape,  elevation,  surface 
characteristics  of  the  surrounding  skin,  and  the  evolution  history  of  these  features,  need  to 
be  analyzed.  Fig.  1-4  shows  two  pictures  of  a malignant  melanoma.  The  left  one  is  the 
initial  phase  picture,  and  right  one  corresponds  to  the  tumor  phase.  Fig.  1.5  indicates  the 
typical  evolution  of  a melanoma.  It  diagrams  melanoma's  features:  shapes,  sizes,  and  color 
distributions,  in  the  different  phases.  These  features,  however,  are  often  poorly  defined 
and  lack  consistency  in  feature  identification;  they  are  not  reliably  identified  by  different 
observers,  even  by  experienced  dermatologists.  To  increase  the  consistency  of  the 
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diagnosis  and  to  provide  a diagnostic  tool  for  many  users  so  that  they  could  evaluate  their 
skin  lesions  conveniently  and  consistently,  a computer  diagnostic  system  is  needed. 

The  diagnostic  system  is  an  image-analysis-based  knowledge  base  system  working 
under  PC  Windows  environment.  Fig.  1-6  shows  the  functional  system  diagram  of  the 
system.  Its  information  sources  are  color  pictures  from  scanners  or  cameras.  The  system 
first  isolates  disease  lesions  by  using  an  image  segmentation  technique.  Based  on  the 
segmentation  results,  a group  of  high  level  analyses  is  carried  out  to  extract  the  visual 
features  of  the  lesion.  Then  these  features  are  classified  into  conceptual  feature  patterns  by 
the  pattern  analyzer.  Finally,  the  feature  pattern  triggers  corresponding  rules  in  the  rule 
base  to  make  a clinical  decision. 

In  such  a system,  image  segmentation  is  the  most  important  step.  High  quality 
segmentation  results  will  better  define  visual  features  of  pictures,  thus  providing  a 
complete  and  consistent  model  for  any  further  analysis.  By  using  an  ensemble  of  features, 
one  can  obtain  higher  levels  of  confidence  than  the  inherent  accuracy  of  any  one  feature. 
Thus,  we  can  increase  the  confidence  in  classification  of  images  which  possibly  contain 
malignant  melanoma.  High  quality  segmentation  should  result  in  uniformity,  compactness, 
dissimilarity  regions,  as  well  as  accurate  and  continuous  boundaries  from  noisy  images. 
Automatic  image  analysis  for  the  detection  of  malignant  melanoma  depends  to  a large 
extent  on  the  approach  for  segmenting  suspect  lesion  regions. 

The  image  segmentation  in  the  system  is  also  the  most  difficult  step  because  (1) 
many  control  parameters  need  to  be  determined  and  it  is  impossible  to  choose  a set  of 
parameters  that  are  suitable  for  all  areas  of  the  image;  (2)  the  control  parameters  interact 
in  complex  ways  and  are  difficult  to  model  mathematically;  (3)  the  variations  of  skin 
lesion  images  cause  the  parameters  and  computations  of  segmentation  quality  to  change 
for  every  image,  and  (4)  various  types  of  noise  introduce  various  kinds  of  unpredictable 
false  elements  into  the  images.  At  present,  no  general  methods  have  been  found  that 
perform  adequately  across  the  diverse  types  of  imagery  encountered  in  suspected 
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Fig.  1-1  Diagnosed  Cancer  Frequency  Curve  in  U.  S.  A.  in  1991 
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Fig.  1 -2  The  Lifetime  Risk  for  an  Individual  American  to  Develop 
Malignant  Melanoma 


I 


1950 


1960 


1970 


1980 


1990 


Fig.  1-3  Death  Rate  and  Incidence  Rate  of  Malignant  Melanoma 
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Fig.  1-4  Pictures  of  a Malignant  Melanoma 
The  left  one  shows  the  initial  phase. 
The  right  one  shows  tumor  phase. 
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Fig.  1-5  Visual  Features  of  Malignant  Melanoma 
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Fig.  1-6  The  Flow  Chart  of  the  System 
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melanoma  pictures  and  no  algorithms  have  been  found  which  resolve  all  difficulties 
satisfactorily. 

The  importance  and  difficulties  of  the  image  segmentation  present  a challenge  for 
developing  a suitable  segmentation  approach  for  the  system.  The  word  "suitable"  means 
that  the  segmentation  approach  for  the  system  must  satisfy  the  following  requirements: 

1 . It  must  be  universally  suitable  to  the  class  of  application  images  without  difficulty 
for  user  input  of  processing  parameters. 

2.  It  must  have  high  performance,  that  is,  be  robust  to  noise  and  generate  high  quality 
segmentation  results  in  a reasonable  time. 

1.2  Objective 

Thus  we  face  two  problems.  First  is  the  development  of  an  approach  that  satisfies 
the  requirements  listed  above.  Second  is  the  determination  of  how  well  that  approach 
works.  That  is,  does  the  newly  developed  procedure  produce  correct  and  consistent 
results  at  a high  performance  level?  The  goal  of  this  dissertation  is  to  address  both  of 
these  problems. 

The  specific  objectives  of  this  study  are  as  follows: 

1 . Develop  a new  approach  satisfying  the  requirements. 

2.  Implement  the  approach. 

3 . Compare  the  proposed  approach  experimentally  with  other  accepted  approaches  to 
show  that  the  performance  of  the  proposed  approach  quantitatively  and 
qualitatively  outperforms  these  established  procedures. 

1.3  Dissertation  Organization 

The  dissertation  consists  of  two  parts:  Part  I --  literature  review,  and  Part  II  — the 
developed  approach.  Part  I reviews  the  literature  on  some  segmentation  approaches. 
Chapter  2 reviews  basic  segmentation  algorithms  and  Chapter  3 presents  advanced 
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segmentation  procedures.  Chapter  4 summarizes  the  main  problems  of  these  approaches. 
Part  II  describes  the  approach  developed  in  this  work,  which  is  a hybrid  segmentation 
approach.  Chapter  5 introduces  the  basic  regularization  theory  from  which  a new 
algorithm  used  in  the  first  stage  of  the  hybrid  segmentation  approach  is  developed.  The 
three  chapters  after  Chapter  5 describe  three  stages  of  the  hybrid  segmentation  approach 
developed  in  this  dissertation.  Then  experimental  results  are  presented  in  Chapter  9,  which 
evaluate  the  developed  approach  and  illustrate  its  ability  to  segment  malignant  melanoma. 
The  final  chapter  summarizes  this  study  and  suggests  directions  for  future  research. 


PART  I 


LITERATURE  REVIEW 


Introduction 


Image  segmentation  is  typically  the  first  key  step  and  also  the  most  difficult  step  in 
processing  an  image.  The  primary  function  of  image  segmentation  is  to  extract  objects  of 
interest  from  any  background  present  in  an  image  by  partitioning  the  image  into 
homogenous  regions,  each  of  which  is  then  processed  further.  Region  homogeneity  could 
be  in  the  sense  of  intensity,  texture,  or  surface  type.  In  segmenting  malignant  melanoma, 
we  typically  segment  on  intensity  in  color  space.  Over  the  last  decades  many  researchers 
have  developed  excellent  segmentation  approaches.  Powerful  algorithms  have  been  used 
in  practice.  Performance  of  segmentation  algorithms/approaches  is  usually  evaluated  for 
the  following  performance  measures: 

. Uniformity — regions  should  be  uniform  or  homogeneous; 

. Compactness — regions  should  be  simple  and  without  small  holes; 

. Dissimilarity — adjacent  regions  should  be  significantly  different; 

. Boundaries  should  be  continuous  and  accurate  . 

As  shown  in  Fig  2-1,  segmentation  techniques  can  be  classified  as  either  statistical 
or  non-statistical  ones.  Under  them,  there  are  three  basic  schemes:  edge/boundary  based, 
region  based,  and  shape-based  segmentation. 

The  statistical  segmentation  techniques  use  statistical  measures  and  analyses  in 
image  segmentation.  Generally  speaking,  statistical  segmentation  approaches  are  less 
sensitive  to  noise  than  non-statistical  ones  since  the  statistical  segmentation  techniques 
employ  information  of  an  area  instead  of  an  individual  point,  and  the  noise  can  be 
adequately  characterized  in  statistical  terms  and  be  imbedded  in  the  statistical  test 
procedures. 
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Edge  based  segmenters  segment  images  by  (1)  using  edge  detection  techniques  to 
find  edges  in  an  image,  (2)  connecting  edges  to  form  closed  contours  by  using  some 
connection  algorithms,  and  (3)  mapping  contours  to  regions.  The  key  step  in  this  kind  of 
segmenters  is  to  find  true  edges.  Region  based  segmenters  segment  an  image  by  grouping 
'similar'  pkels  to  form  region  patches.  The  variations  among  the  region  based 
segmentation  techniques  lie  in  the  grouping  schemes  and  the  definitions  of  the  similarity. 
Shape  based  segmenters  use  morphological  operators  with  certain  shape  structures  in 
noise  removing,  edge  detection  and  region  reshaping. 
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Fig.  2-1  Segmentation  Technique  Classification 


CHAPTER  2 

BASIC  SEGMENTATION  ALGORITHMS 
21.  Edge  Based  Segmentation 

In  edge  based  segmentation,  edge  detector  procedures  are  the  most  important  ones 
since  edges  from  the  edge  detectors  are  the  basic  elements  of  the  boundaries  of  objects,  or 
of  the  regions  which  are  to  be  segmented.  The  most  popular  classical  edge  detectors  are 
derivative  edge  detectors  which  can  be  classified  as  first  order  derivative  edge  detectors 
and  second  order  derivative  edge  detectors  [Gon  87],  Both  of  these  processes  are  non- 
statistical  ones. 

2.1.1.  First  Order  Derivative  Edge  detectors 

The  first  order  derivative  edge  detectors  label  a pixel  as  an  edge  pixel  if  the 
amplitude  of  its  gradient,  dG(i,  j),  is  larger  than  a threshold,  T.  Thus  the  edge  pixel  set,  E, 
is  defined  as 

E = {pixel ^^(x,y),  (x,y)  e -^^^^cosn+  -^^^^sinQ>  T}  (2-1) 

This  idea  has  been  implemented  by  Roberts,  Prewitt,  Sobel  and  Frei-Chen  as 
follows.  Let  F=(fj,  f2,  ...  , f9)  be  a vector  containing  the  pixel  values  of  a 3*3 
neighborhood  shown  in  Fig.  2-2(a);  Wplwjj,  wj2,  be  the  vector  representing  the 

mask  shown  in  Fig.  2-2  (b); 

W = Jz(w.F)'  . (2-2a) 
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Fig.  2-2  The  operator  of  the  first  order  derivative  edge  detector 
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Thus 

an  edge  pixel  if  W > T 

f5  is  { (2-2b) 

a non-edge  pixel  otherwise 

The  larger  the  T,  the  less  sensitive  the  edge  detector  is  to  indicating  that  an  edge  exists. 
The  differences  among  the  schemes  attributed  to  Roberts,  Prewitt,  Sobel  and  Frei-Chen  lie 
in  the  values  of  Wj  and  m (see  Fig.  2-2)  [Rob  65][Pre  70][Gon  87][Fre  75].  These  first 
order  derivative  edge  detectors  are  the  simplest  to  implement. 

Any  edge  detectors  involving  derivatives  are  necessarily  very  sensitive  to  noise.  To 
handle  noisy  cases,  noise  removal  processing  is  needed  before  edge  detection.  Noise 
removing  can  be  done  by  many  kinds  of  linear  low-pass  filters  and  non-linear  filters. 
Linear  filters  include  moving  average  filters,  Gaussian  filters,  Butterworth  filters  and 
Homomorphic  filters.  An  example  of  a non-linear  filter  is  the  median  filter.  Canny  [Can 
83]  has  shown  that,  in  1-D,  the  optimal  filter,  according  to  his  criterion,  is  a linear 
combination  of  four  exponentials,  which  can  be  well  approximated  by  a first  derivative  of 
a Gaussain.  He  has  used  a filter  combined  by  such  filters  with  varying  size  and 
orientations,  first  convolving  it  with  the  original  images  and  then  thresholding,  to  detect 
edges  of  2-D  images. 

The  edges  from  the  first  order  edge  detectors  are  typically  thick  and  not  closed, 
since  they  indicate  individual  pixels  as  being  'an  edge  point'.  A group  of  adjacent  edge 
pixels  will  form  a boundary,  an  edge  segment,  of  a region  in  the  image.  What  we  need  is  a 
definite  boundary,  a curve  representing  the  edge  of  a region,  not  a set  of  disjoint  points 
which  may  form  clumps  and  thus  indicate  a thick  region  for  the  separation  curve.  This 
means  further  thinning  and  connecting  are  required.  The  threshold  value,  T,  and  filter  size 
will  greatly  affect  edge  detection  quality. 
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2. 1 .2.  Second  Order  Derivative  Edge  Detectors 

Second  order  derivative  edge  detectors  locate  edges  of  non-noisy  images  by  noting 
the  zero  crossing  of  the  second  order  directional  derivative.  That  is,  the  edge  pixel  set  is: 


E = {pixeledge(x,  yX  (x,  y)  e 91} 


(2-3a) 


91=  { -V^f(x'^,y^)>  0 and 
or  -V^f(x'^,y^)^  0 and 
or  -V^f(x‘^,y^)^  0 and 


V^  = 


d'^ 


+ 


-v^f(x  ,y  0 
-v"f(x^y■)^  0 

-v^f(x-,y^)^o} 


(2-3b) 


(2-3c) 


To  handle  noise  cases,  a Gaussian  filter  with  size  X,  may  be  used  [Mar  80.]  Marr 
and  Hilderth  first  undertook  Laplacian  operation  on  a Gaussian  filter  to  form  a rationally 
symmetrical  Laplacian  of  Gaussian  (LoG)  mask.  Then  they  used  the  LoG  mask  instead  of 
the  Gaussian  filter  for  convolution  with  images.  Marr  and  Hilderth's  contribution  led  to 
one  of  the  most  popular  edge  detectors,  the  LoG  edge  detector.  The  LoG  edge  detector 
generates  closed  and  one-pixel  wide  edges.  The  main  problems  associated  with  the  LoG 
edge  detector  are  as  follows:  (1)  For  some  edge  patterns,  the  detectors  with  some  range  of 
X will  generate  zero  crossings  at  reflection  points  rather  than  at  real  edge  points,  that  is  the 
zero  crossing  does  not  always  correspond  to  real  edge  points;  (2)  There  exists  a shift 
between  zero  crossing  and  actual  edge  position  because  of  the  interaction  between  nearby 
edges;  and  (3)  It  is  difiScult  to  obtain  good  performance  on  both  edge  position  and  edge 
detection.  The  tradeoff  between  edge  position  and  edge  detection  is  controlled  by  the  size 
of  the  Gaussian  filter.  A small  X gives  rise  to  more  accurate  edge  position,  but  also  more 
false  edges.  A large  X can  reduce  the  false  edges,  but  the  edge  position  is  shifted  and  some 
true  edges  are  missing  (or  edge  detection  ability  decreases).  It  is  impossible  to  choose  a X 
that  simultaneously  suppresses  noise  and  retains  the  details  in  the  object  boundaries.  To 
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solve  these  problems,  extensive  research  has  already  been  done,  but  without  useful  results 
for  our  work. 

Chen  and  Medioni  have  derived  the  mathematical  formula  for  judging  false  zero 
crossing  and  correcting  the  bias  of  edge  positions  [Che  87].  They  point  out  that  a zero 
crossing  corresponds  to  an  edge,  if  and  only  if,  f ' * f "<  0 and  the  gradient  magnitude  is 
larger  than  a threshold.  It  happens,  however,  that  some  valid  edges  may  be  missed  when 
the  X is  so  large  that  f ' will  produce  the  wrong  sign,  or  if  an  unsuitable  threshold  is  used. 

Raman's  multiresolution  scale  space  tracing  algorithm  [Ram  91]  detects  edges  in 
different  Xs.  These  X,s  form  a scale  space.  The  algorithm  first  detects  edges  at  a coarse 
scale  (which  gives  few  false  edges),  then  follows  their  progress  through  the  scale  space, 
and  tracing  the  contour  through  the  scale  space.  Contours  are  refined  starting  with  coarse 
values,  which  are  then  successively  refined.  The  problems  with  this  technique  are  that  the 
coarse  scales  have  already  introduced  inaccurate  edge  positions  and  that  multicell  edge 
detecting  requires  extended  computer  time  and  large  storage  capacity. 

Lee's  algorithm  [Lee  91a]  first  detects  edge  points  at  small  scale  space,  getting  an 
intermediate  edge  image.  Then  the  system  checks  each  edge,  deleting  the  ones  whose 
length  is  below  a threshold.  For  the  rest  of  the  edges,  his  technique  calculates  the 
edges'  curvature  p(x),  estimates  their  gradient  variance  V(x)  and  computes  a weight 
defined  as  ^cU(m(x))V(i:)dx  Finally,  the  algorithm  deletes  all  edges  with  W(C) 

below  a threshold.  This  algorithm  can  obtain  rather  good  results  with  extensive 
computation  and  good  threshold  settings. 

To  eliminate  some  false  zero  crossings,  an  alternative  zero  crossing  approach  was 
developed  by  Lee  [Lee  91b].  It  is  called  the  residual  analysis  edge  detection.  In  this 
approach,  the  differentiation  operation  is  replaced  by  residuals  which  are  equal  to  the 
difference  between  the  original  image  and  its  smoothed  version.  The  zero  crossing  of  the 
residuals  yields  edges  provided  that  on  either  side  the  sign  of  the  residual  is  constant  for 
some  area.  A,  and  the  gradient  of  the  residual  is  significant  [Gri  85].  By  residual  analysis. 
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the  false  zero  crossings  generated  by  triangular  creases  can  be  ignored  since  the  gradient 
of  the  residual  is  very  small.  Lee  proposed  two  criteria  for  further  eliminating  false  zero 
crossings.  The  first  criterion  states  that  those  zero  crossings  with  which  two  associated 
regions,  R'*'  and  R’>  are  not  both  simply  connected  are  considered  as  false  ones.  The 
second  criterion  computes  the  a quantity  at  zero  crossing  (xq,  yo), 

2 2 
M r (><0.  Vo)  = [4t>«  - xo)[  f(x . y)  - d(x . y)]dxdy] +[yy  - yo)[  f(x . y)  - d(x . y)]dxdyj 

and  checks  Mj^Xq,  yo)  against  a threshold.  If  Mj^Xq,  yo)  is  larger,  the  zero  crossing  is 
kept,  otherwise  it  is  removed.  Although  the  residual  edge  detector  can  eliminate  some 
false  edges,  it  has  not  relieved  the  last  two  problems  of  LoG  edge  detectors  mentioned 
above. 

2.1.3.  Statistical  Edge  Detection 

The  difference  between  statistical  and  non-statistical  edge  detectors  lies  in  the 
criterion  of  labeling  an  edge  point.  The  statistical  edge  detectors  designate  edge  pixels 
based  on  results  of  statistical  tests  in  the  pbcel's  neighborhood,  called  the  window. 

One  of  such  edge  detector  was  published  by  Kundu  [Kun  89].  In  that  work,  he 
defined  three  types  of  regions:  quasi-constant  (QC),  large  linear  varying  (LLV)  and  quasi- 
two-value  (QTV),  and  assumed  the  population  of  pixels  inside  each  region  has  Gaussian 
distribution  with  mean,  |i,  and  small  standard  deviation  for  QC  or  a large  one  for  LLV.  As 
the  window  of  size  n*n,  centered  at  pixel(i,  j)  moves  over  the  image  in  a raster  scan 
fashion,  three  statistical  testing  steps  are  applied  on  each  pixel.  An  edge  candidate  pixel 
must  pass  the  three  tests  described  in  the  following  three  steps. 

• Step  1:  (1)  testing  the  dispersion  of  noise  statistics 

XP_i  r =>  non-QC  region; 


(2)  thinning  process 
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Xo^ 


X"  1+X5 


having  edges  not  wider  than  2 pixels 


where  p = n * n and  Xf  is  the  i^“  element  of  the  ordered  pixel  value 
set  of  the  n * n window,  X = {xf , X5, ...  Xf_i , Xf } 

Step  2:  (1)  dividing  the  passed  non-QC  region  with  a increased  size,  R = (n+dn)  * 
(n+dn)  into  two  groups  based  on  the  2^"  order  statistic 

Xi  ^ ^ =>  Xj  belongs  to  group  L with  mean  pj 


otherwise 


X}  belongs  to  group  H with  mean 
(i=l,2,...P) 


(2)  testing 


T = |xh-xil+lyh-yil^;5  =>  non-QC  region 


where  (xjj,  yj^)  and  (xj,  yj)  are  the  coordinates  of  p^  and  pj  respectively. 

• Step  3:  Testing  on  the  non-QC  region  passed  step  2 

I stat|=  ^ ^ TJ  =>  pixel  (i,  j)  is  an  edge  pkel 

where  Bj  = bi+b2+ +bR.  bj  e (0,  1)  and  is  a Bernoulli  variable  with 

probability  1/2. 

There  are  still  many  spurious  edges  and  missing  edges  after  the  three  tests  detailed  above, 
so  the  algorithm  uses  another  non-statistical  test  on  the  window  structures  of  2D 
hypothesized  contour  segments.  The  test  calculates  a quantitative  amount,  called  U- 
number,  for  every  hypothesized  contour  segment,  selects  the  largest  U-number,  Umax* 
and  tests  whether  or  not  Umax  is  larger  than  a threshold  If  so,  the  central  pixel  is  a part 

of  a valid  edge  contour.  Otherwise  it  is  a spurious  edge  pixel.  The  edge  detector  works 
fairly  well  with  properly  threshold  setting. 

Another  statistical  edge  detector  was  developed  by  Suk  [Suk  84].  The  outline  of 
the  algorithm  is  shown  in  Fig.  2-3.  The  two  most  important  steps  of  the  algorithm  are 


22 


modality  test  and  edge  detection.  The  modality  test  is  the  heart  of  the  algorithm.  The 
modality  test  means  (1)  for  every  pixel  in  the  testing  window,  several  statistical  tests,  such 
as  simple  parametric  test,  test  based  on  the  gray  level  distribution,  and  test  based  on  the 
homogeneity  of  spatial  variance,  are  applied;  (2)  for  spatial  variance  test,  a number  of 
different  methods,  such  as  ANOVA,  shortcut  method.  Maximum  F ratio  and  Cochran 
methods  [Suk  84],  are  employed.  Such  redundancy  tests  increase  the  reliability  and 
consistency  of  detection  because  the  decision  on  the  existence  of  a significant  connected 
edge  segment  in  the  window  is  made  by  examining  all  of  the  test  results.  Therefore,  even 
though  some  of  the  tests  produce  errorous  results  due  to  noise  effects,  the  final  decision 
based  on  the  results  of  all  tests  has  a much  higher  probability  of  being  correct.  During  the 
modality  test,  a "no-decision"  category  is  added  to  "acceptance"  or  "rejection"  of  the 
hypothesis  since  it  is  more  reasonable  to  allow  a "no  decision"  than  to  force  a definite 
decision  when  in  doubt.  After  the  modality  test,  the  final  decision  scheme  on  determining 
whether  a window  contains  a connected  edge  segment  is  the  majority  vote  approach  to  all 
of  the  test  results.  For  each  edge-containing  window,  the  histogram  thresholding  and  edge 

t 

detection  are  used  to  find  the  edge  segment  position.  In  this  edge  detector,  the  window 
moving  scheme  is  block  by  block  rather  than  in  a raster  format,  or  pixel  by  pixel. 
Therefore,  the  size  of  the  window  is  a critical  parameter.  Valid  sizes  are  only  those  that 
are  small  enough  so  that  the  edge  segment  can  cross  the  window  and  large  enough  to 
guarantee  the  validity  of  statistical  tests.  Of  course,  all  thresholds  in  every  test  step  also 
affect  the  performance  of  the  detector. 

2.2.  Region  Based  Segmentation 

Region  based  segmenters  segment  an  image  by  grouping  'similar'  pixels  to  form 
region  patches.  The  basic  principle  is  described  as  follows. 

Let  R represent  the  entire  region.  We  may  view  segmentation  as  a process  that 


partitions  R into  n subregions,  Rj,  R2,....,Rn,  such  that 


input  image 


output  image 


Fig.  2-3  Suk's  Statistical  Edge  Detector 


24 


n 

a)  URi  = R 
i=l 

b)  Ri  is  a connected  region,  i=l,2,....n, 

c) Rjr^Rj=d>  forV  i and  j,  i^^j 

d)  P(Ri)  = true  for  i=l,2,...,n, 

e)  P(RjuRj)  = false  for  i 

Where  P(Ri)  is  a logical  predicate  defined  over  the  points  in  set  Ri,  and  O is  the  null  set. 

Condition  (a)  indicates  that  the  segmentation  must  be  complete.  Condition  (b) 
requires  that  the  points  in  a region  must  be  connected.  Condition  (c)  indicates  that  the 
regions  must  be  disjoint.  Condition  (d)  deals  with  the  properties  which  must  be  satisfied  by 
the  pixels  in  a segment  region.  For  example,  P(Ri)  can  be  some  measure  of  uniformity. 
Condition  (e)  indicates  that  region  Ri  and  region  Rj  are  different.  The  variation  of  the 
region  based  segmentation  lies  in  the  different  definitions  of  P(Ri)  and  the  methods  to 
measure  P(Rj).  Four  classical  approaches  are:  histogram  thresholding,  region  growing, 
region  splitting  and  merging,  and  clustering  approach  [Gon  87]. 

2.2.1.  Histogram  Thresholding  Methods 

In  the  image  histogram  domain  objects  can  be  represented  by  the  peaks  in  the 
image  histogram  if  the  image  is  noiseless.  Therefore  histogram  thresholding  techniques 
offer  a fast  and  straightforward  approach  to  extract  objects  fi’om  the  background.  One  can 
pick  out  the  objects  by  setting  suitable  thresholds,  Tj,  (see  Fig.  2-4  (c)).  The  following 
introduces  several  histogram  thresholding  algorithms. 

Global  histogram  thresholding  techniques  are  based  on  histogram  analysis  of  the 
entire  image.  They  are  suitable  for  images  whose  histograms  have  well-defined  peaks.  The 
thresholds  are  located  at  valleys  between  peaks.  A popular  method  for  finding  the 
thresholds  is  optimal  thresholding  [Gon  87],  in  which  the  thresholds  are  set  such  that 
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Fig.  2-4  Image  histogram 
(a)  Original  image,  (b)  Histogram  of  (a). 

(c)  A histogram  of  well-defined  peak  characteristics. 

(d)  A histogram  of  pool-defined  peak  characteristics. 
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either  the  overall  probability  of  error  or  the  mean-square  error  between  segmented  image 
and  the  original  image  is  minimum.  This  kind  of  optimal  problem  can  be  solved  by 
traditional  numerical  methods,  such  as  Newton's  method.  A more  analytical  method  is  (1) 
fitting  the  histogram  by  N-modal  Gaussian  density  curve  where  N = the  number  of  peaks 
in  the  histogram,  or  fitting  the  i^h  peak  by  the  a Gaussian  distribution  with  mean  pj  and 
standard  deviation,  oj;  (2)  calculating  the  thresholds,  Tj,  between  the  i^"  peak  and  (i+1)*” 
peak,  using  the  following  formula 


2 


(2-5) 


where  pj  is  a priori  probability  of  peak  i.  Thus  the  pixels  with  gray  level  between  Tj.j  and 
Tj  are  classified  to  the  i^"  segment,  labeling  as  Rj  (i=l,2,  ...  K,  and  K is  the  assumed 
number  of  segments).  This  method  is  suitable  for  the  images  having  histograms  with  well 
separable  peaks  and  approximate  Gaussian  distribution.  (Note  that  Gaussian  distribution 
histogram  is  assumed  by  many  published  papers,  for  example,  in  the  papers  cited  earlier  on 
the  two  statistical  edge  detectors).  Many  images,  however,  are  quite  different.  This 
dissertation  will  describe  a new  algorithm  for  finding  the  optimal  thresholds. 

Another  global  thresholding  algorithm  was  developed  by  Amulfo  and  Gonzalez 
[Am  87].  It  segments  a real  image  to  a binary  object-background  image  by  iteratively 
updating  the  segmentation  threshold  based  on  the  Taylor  series  expansion  in  a raster 
format.  First,  the  algorithm  uses  Stockham's  model  to  describe  a visual  gray  level  image, 
f(x,  y)  as  the  product  of  an  illumination  component,  I(x,  y),  and  a reflectance  component, 
r(x,  y).  Second,  in  order  to  dynamically  segment  the  image  under  varying  lighting 
condition,  the  threshold  is  expressed  as  a recursive  function  of  f.  Third,  under  the 
assumption  that  changes  in  I(x,  y)  are  smooth,  for  example  linear  or  Gaussian  illumination 
function,  and  that  abmpt  changes  in  f are  due  to  r but  with  r values  clustered  in  areas  of 
constant  reflectivity,  the  recursive  function  can  be  expanded  as  a Taylor  series.  Thus  at 
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each  pixel  site,  a corresponding  threshold  with  an  arbitrary  initial  value  is  iteratively 
modified,  approaching  a better  and  better  value.  At  last,  every  pixel  in  the  image  is 
assigned  to  either  object  or  background  according  the  final  threshold  at  corresponding 
sites. 

The  problems  with  such  a global  approach  include  the  overlapping  distributions 
that  allow  hidden  objects  and  the  loss  of  spatial  information.  Thus  localized  histogram 
thresholding  approaches  are  used.  The  localized  histogram  algorithms  divide  a whole 
image  into  many  small  subimages  and  employ  the  histogram  thresholding  techniques  over 
each  localized  histogram  to  label  pixels  to  produce  a segmentation.  The  size  of  the 
subimages  varies  depending  on  the  different  algorithm.  For  example,  in  Chow  and 
Kaneko's  algorithm  [Cho  72],  the  window  size  is  chosen  such  that  the  subwindow's 
histogram  is  bimodal  so  that  the  Gaussian  curve  fitting  method  can  be  used  to  find  the 
optimal  threshold.  One  of  the  problems  with  the  localized  histogram  is  that  the  artificial 
subimage  boundaries  are  induced  into  the  segmented  image.  To  solve  this  problem, 
Beveridge  developed  an  algorithm  [Bev  89].  In  his  algorithm,  the  subimage  size  is  16x16 
with  4-pixel  overlap  expansion  in  each  direction,  to  correct  situations  in  which  a subimage 
boundary  happens  to  divide  an  area  in  two,  where  a single  region  should  be.  The  algorithm 
begins  with  first  identifying  all  local  maximals  in  the  corresponding  localized  histogram  as 
peaks  and  the  points  of  minimum  value  between  each  adjacent  pair  of  peaks  as  a valley, 
then  applying  three  constraints  to  remove  those  peaks  which  have  peak  height  < a , peak- 
valley  ratio  < P,  or  peak  distance  < y (a,  P,  or  y is  a threshold)  where, 
peak  height  = the  magnitude  of  the  histogram  at  the  peak, 
peak  distance  = the  gray-level  difference  between  two  peaks, 
peak-valley  ratio  = the  peak  height  / the  higher  of  the  two  neighboring  valleys. 

The  valleys  satisfying  the  three  constraints  correspond  to  those  thresholds  in  the  histogram 
which  are  used  for  thresholding  the  subimage  into  segments.  To  remove  artificial 
boundaries  between  subimages,  all  pairs  of  regions  adjacent  at  a subimage  boundary  are 
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compared  using  both  a local  and  a global  measure  of  similarity,  called  merge-score.  The 
merge-score  is  defined  as 

merge  - score  = — — — (2-6) 

max(<Ta  + o^, , 1) 

where  |x  is  the  mean  and  a is  the  standard  deviation.  For  global  measure,  all  pixels  in  the 
respective  regions  contribute  to  the  computation  of  p,  and  a.  For  local  measure,  only 
pixels  within  a fixed  distance  (usually  4 pixels)  of  the  shared  boundary  are  considered. 
Two  regions  are  merged  if  and  only  if  both  the  local  and  global  merge-scores  are  below 
the  specified  thresholds. 

2.2.2.  Region  Growing  Method 

Region  growing  is  a computationally  simple  and  conceptually  sophisticated  method 
for  image  segmentation.  Region  growing  algorithms  segment  images  by  grouping  pixels  or 
subregions  into  large  regions  on  the  basis  of  some  similarity  criteria.  The  simplest  of  them 
is  the  pixel  arrogation  algorithm  [Gon  87].  The  algorithm  starts  with  a set  of 'seed'  points 
and  fi'om  these  grows  regions  by  appending  to  each  seed  point  those  neighboring  pixels 
that  have  similar  properties  (e.g.,  gray  level,  texture,  color).  Two  important  problems  are 
the  selection  of  initial  seeds  that  properly  represent  regions  of  interest,  and  the  selection  of 
suitable  properties  for  including  points  in  the  various  regions  during  the  growing  process. 
The  resolutions  of  these  two  problems  lead  to  many  variations  of  the  region  growing 
approach. 

Usually,  seed  selection  needs  a priori  information  on  the  nature  of  the  problem. 
When  a priori  information  is  not  available,  one  may  proceed  by  computing  at  every  pixel 
the  same  set  of  properties  that  will  ultimately  be  used  in  assignment  of  pbcels  to  regions 
during  the  growing  process.  If  the  result  of  this  computation  shows  clusters  of  values,  then 
the  pixels  whose  properties  place  them  near  the  centroid  of  these  clusters  can  be  selected 
as  seeds. 
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The  selection  of  similarity  criteria  is  dependent  not  only  on  the  problem  under 
consideration,  but  also  on  the  type  of  image  data  available.  For  example,  the  analysis  of 
land-use  satellite  imagery  is  heavily  dependent  on  the  use  of  color.  Often,  a number  of 
similarity  measures  are  used  simultaneously  on  a single  image.  Besides  the  similarity 
criterion,  connectivity  or  adjacency  information  should  be  used  in  the  region-growing 
process  to  prevent  a meaningless  result.  One  example  of  the  use  of  such  information  is:  (1) 
the  absolute  difference  in  gray  level  between  the  seed  and  a candidate  pixel  must  not 
exceed  1 0%  of  the  difference  between  the  minimum  and  maximum  gray  levels  in  the  entire 
image;  and  (2)  any  pixel  added  to  the  region  must  be  8-connected  to  at  least  one  pixel 
previously  included  in  the  region. 

The  thresholds  used  in  testing  the  similarities  also  affect  the  segmentation  results. 
For  example,  the  number  of  the  segmented  regions  is  at  most  equal  to  the  number  of 
seeds,  depending  on  the  threshold  value.  Even  the  priority  of  growing  directions  will  affect 
the  shape  of  regions. 

2.2.3.  Region  Split  and  Merging  Methods 

Unlike  region  growing  methods,  region  split  and  merging  approaches  segment 
images  by  splitting  those  regions  whose  P(Ri)  = false  and  merging  those  whose  P(RjuRj) 
= true.  One  more  practical  example  of  split  and  merge  algorithm  has  been  implemented  by 
initially  subdividing  an  image,  R,  into  a set  of  arbitrary,  disjointed  regions,  then  splitting  a 
region  if  the  variance  of  the  region  is  larger  than  a threshold,  or  merging  two  adjacent 
regions  if  the  contrast  of  two  regions  is  smaller  than  a threshold,  and  finally  labeling  the 
pixels  in  Rj  as  the  mean  value  of  Rj. 

A popular  algorithm  is  called  quadtree  split  and  merging  algorithm  in  which  the 
geometric  data  structure  is  a quadtree  (the  quadtree  is  a tree  in  which  each  node  has 
exactly  four  descendants)  [Gon  87].  The  algorithm  is  summarized  as  follows: 

(1)  The  initial  region  Rq  = whole  image. 
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(2)  Split  any  region  Rj  into  four  disjointed  quadrants  where  P(Ri)  = false. 

(3)  Merge  any  adjacent  regions  Rj  and  Rj  for  which  P(Rj  uRj)  = true. 

(4)  Stop  when  no  further  merging  or  spliting  is  possible. 

A simple  illustration  on  the  algorithm  is  shown  in  Fig  2-5.  One  problem  of  this  algorithm  is 
that  the  rectangular  tessellation  of  the  image  domain  often  results  in  some  false  region 
boundaries.  To  remove  the  false  boundaries,  either  the  procedure  of  identifying  and 
removing  false  boundaries  is  needed,  or  another  data  structure  of  a more  geometric 
flavor  is  used.  For  example,  in  Schmitt  and  Chen's  segmenter,  a triangle  structure  is  used 
in  tessellating  the  image  domain  [Sch  91]. 

2.2.4.  Clustering  Approach 

The  cluster  techniques  may  also  be  used  in  region  based  segmentation.  In  this  case, 
the  segmentation  procedure  can  be  considered  as  grouping  the  pixels  with  similar 
properties  together  by  finding  the  optimal  cluster  center  and  assigning  each  pixel  to  the 
closest  cluster  center.  In  this  kind  of  approach,  the  selection  of  similarity  measure  and  how 
to  find  the  optimal  cluster  centers  lead  to  many  algorithms,  such  as  K-mean,  Max-Min, 
etc.  [Tou  77],  algorithms.  In  fact,  the  histogram  thresholding  techniques  can  be  seen  as  a 
kind  of  cluster  technique  in  the  histogram  domain. 

Usually,  the  region  based  algorithms  are  not  robust  to  noise.  In  noisy  cases,  the 
segmentation  results  are  uncompact,  namely  there  are  many  small  holes  in  the  segmented 
images.  Therefore,  preprocessing  (e.g.  noise  removing)  or  postprocessing  (e.g.,  false 
region  removing)  is  necessary. 

2.3.  Shape  Based  Segmentation— (Morphology  Methods! 

Since  Serra  and  Sternberg  introduced  mathematical  morphology  to  low-level 
image  processing  [Ser  82]  [Ste  86],  many  useful  image  processing  procedures  using  basic 
operators  of  mathematical  morphology  have  been  developed.  Morphological  image 
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processing  is  a shape-oriented  one.  It  tends  to  simplify  image  data  by  preserving  the 
essential  shape  characteristics  and  eliminating  irrelevancies.  It  treats  the  image  as  a set, 
and  applies  different  set  operations,  such  as  dilation,  erosion,  opening  and  closing,  and 
some  basic  image  processing  methods,  like  contrast  enhancement,  histogram  equalization, 
etc.,  in  low-level  image  processing.  Several  research  achievements  in  this  area  are 
described  below. 

2.3.1  Morphological  Seementer 

Klinger  [Kli  88]  and  Thomas  [Tho  91]  have  developed  morphological  segmenters 
for  ultrasound  images  (Thomas'  segmenter  was  an  improvement  on  Klinger's  work).  The 
segmenters  generate  shape-oriented  operators  according  to  the  a priori  knowledge  about 
the  general  size  of  image  features  for  removing  interfering  data  based  on  the  image  shape. 
The  Thomas'  segmenter  consists  of  the  following  processing  steps:  (l)the  gray  scale 
opening  with  a disk-shaped  structure  element  of  33  pixels  in  diameter,  which 
morphologically  filters  the  input  image  to  obtain  a slowly  varying  background  image;  (2) 
subtraction,  which  subtracts  the  background  image  from  the  original  image;  (3)  contrast 
enhancement,  which  is  used  to  produce  a higher  contrast  image  from  the  subtracted  image 
since  it  is  usually  fairly  dark;  (4)  thresholding  on  enhanced  image,  which  produces  a binary 
foreground-background  image;  (5)  region  labeling  and  object  recognizing,  out  of  which  is 
selected  the  object  of  interests;  (6)  binary  closing  with  a disk-shaped  structure  of  5 pixels, 
which  is  used  to  fill  small  holes  and  crevices  on  the  object;  and  (7)  binary  opening  with  a 
disk-shaped  structure  of  9 pixels,  whose  purpose  is  to  remove  spurious  extensions  (size  < 
9 pixels  ) from  the  object.  The  performance  of  the  segmenters  is  dependent  of  the 
accuracy  of  the  shape  and  size  of  morphological  structure  elements  which  are  determined 
by  the  prior  knowledge  of  the  application  images. 
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2.3.2  Morphological  Edge  Detector 

Noble  used  morphological  operators  for  feature  detection  of  noise  free  intensity 
images  [Nob  88].  He  detects  edges/boundaries  and  junctions  by  using  the  difference 
operation  of  dilation  and  erosion,  and  the  ridges  and  comers  by  the  difference  of  the  image 
surface  opening  and  closing.  The  element  used  in  the  morphological  operators  is  a 3 x 3 
disk  stmcture.  The  boundary  points  generated  by  the  algorithm  include  stripe  regions 
(smooth  surface  patches),  edges  (singular  curves,  i.e.,  steps,  ridges,  valleys  and  thin  bars) 
and  junctions  (singularity  points).  The  algorithm  is  sensitive  to  noise. 

2.3.3  The  Implementation  of  Morphological  Operators 

In  the  Salembier's  multiresolution  decomposition  and  adaptive  filtering  [Sal  91], 
rank  order  filters  (ROF)  are  used  to  implements  the  gray  scale  morphological  dilation, 
erosion,  opening  and  closing  with  a box-shaped  stmcture  element.  According  to  his 
notation,  if  xj  represents  the  input  signal,  M a mask  containing  N points,  j an  index 
belonging  to  M and  r the  normalized  filter  rank  (0  < r < 1),  the  ROF  output  at  location  i, 
yj,  denoted  as  Rr^Mt^i]>  calculated  by  (1)  taking  all  the  points  xj.j  belonging  to  the  mask 
centered  on  i,  (2)  sorting  their  values  in  ascending  order,  and  (3)  taking  the  n^h  value  (n  = 


r*(N-l)+l)  of  the  ordered  list,  that  is 

y;=Rr,M[xi]=  Rank  (xi-j,jeM)  (2-7) 

n=  r»(N- 1>+1 

Thus  the  morphological  operators  on  xj  are  expressed  as  follows: 

dilation:  Dil  [xi]=Ri  ^[xi]  (2-8a) 

erosion:  Ero  [xj]=Ro  {^^[xi]  (2-8b) 

opening:  Open  [xi]=Ri^M[Ro,M[xi]]  (2-8c) 

closing:  Close[xi]=Ro,M[Rl,M[xi]]-  (2-8d) 


Salembier's  work  provides  a simpler  way  of  implementing  morphological  operators  of 
stmcture  elements. 


CHAPTER  3 

ADVANCED  IMAGE  SEGMENTATION 


Many  basic  segmentation  approaches  have  been  reviewed  in  Chapter  2.  In  this 
chapter,  more  advanced  image  segmentation  approaches  are  described.  These  methods  are 
adaptive,  optimizing,  or  integrating  image  segmentation  algorithms. 

3.1  Adaptive  Segmentation 

One  of  the  keys  to  effective  image  segmentation  is  the  selection  of  the  appropriate 
control  parameters  in  the  segmentation  algorithms  for  each  image.  However,  this  is  very 
difficult  in  most  cases,  but  one  may  try  adaptive  adjustment  of  parameters  based  on  test 
images. 

3.1.1.  Adaptive  Scheme  Based  on  Genetic  Algorithm 

This  adaptive  scheme  is  based  on  the  genetic  algorithm,  in  which  the  selection  of 
control  parameters  is  viewed  as  a search  through  the  hyperspace  of  all  possible  parameter 
combinations.  The  algorithm  determines  the  parameter  set  which  maximizes  the 
segmentation  quality  criteria.  The  genetic  adaptive  approaches  do  not  require  complex 
segmentation  quality  surface  descriptions,  domain  specific  knowledge,  or  measure  of  goal 
distance,  and  are  also  independent  of  the  particular  image  segmentation  technique  which  is 
used  since  they  use  simple  recombination  of  existing  high  quality  parameter  sets  and  a 
method  of  measuring  current  segmentation  quality.  Usually,  genetic  adaptive  systems  are 
more  complicated  in  implementation. 

An  example  using  this  kind  of  adaptive  scheme  can  be  found  in  Bhanu,  Ming  and 
Lee's  paper  [Bha  91].  They  built  an  Adaptive  Image  Segmentation  System  shown  in 
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Fig.  3-1.  The  system  works  as  follows.  (1)  Image  analysis  unit  analyzes  the  image 
characteristics  (first  order  statistics  and  histogram  properties;  a total  of  48  total  image 
properties  from  R,  G,  B,  and  Y components)  and  passes  them  together  with  the  external 
variables  to  the  genetic  learning  unit.  (2)  The  genetic  learning  unit  selects  a possible 
parameter  set  by  comparing  the  current  image  properties  with  those  of  images  processed 
previously  (which  are  stored  in  the  global  population).  The  parameter  sets  form  the  seed 
population  for  the  genetic  process.  This  population  is  maintained  and  updated  by  the 
genetic  learmng  unit.  (3)  Each  of  the  parameter  sets  is  sent  to  the  image  segmentation  unit 
where  images  are  segmented.  (4)  The  segmentation  results  are  evaluated  using  several 
segmentation  quality  measures.  The  evaluated  results  are  combined  and  normalized  in  a 
weighted  sum  that  represents  the  overall  segmentation  quality.  (5)  The  evaluated  results 
are  fed  back  into  the  genetic  unit  as  a "reward"  to  the  associated  parameter  set  and  are 
used  to  adjust  those  parameters.  If  none  of  the  members  in  the  seed  population  is 
acceptable,  the  genetic  learning  unit  begins  recombining  the  parameter  sets  using 
crossover  and  mutation  genetic  operators.  This  adaptive  adjusting  cycle  repeats  until 
acceptable  segmentation  quality  results  have  been  achieved. 

3.1.2.  Knowledge  Directed  Adaptive  Scheme 

Another  adaptive  scheme  is  a knowledge  directed  one.  This  kind  of  scheme  is 
especially  suitable  to  specific  applications  since  prior  knowledge  about  the  images  can  be 
obtained.  Duncan's  algorithm  of  Knowledge  Directed  Left  Ventricular  Boundary 
Detection  in  Equilibrium  Radionuclide  Angiocardiography  [Dun  87],  Michael  and  Nelson's 
Model-Based  System  for  Automatic  Segmentation  of  Bones  from  Digital  Hand 
Radiographs  [Mic  89],  Raya's  Rule-Based  System  for  Low-level  Segmentation  of  3-D 
Magnetic  Resonance  Brain  Images  [Ray  90],  and  Morris'  Rule-Based  System  for 
Dimensional  Analysis  of  Glass  Containers  [Mor  89]  all  belong  to  this  genre.  All  of  these 


36 


% 


Fig.  3-1  An  Adaptive  Image  Segmentation  System 
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algorithms  are  context  dependent,  that  is  the  segmentation  performance  depends  on  prior 
knowledge  of  correct  segmentation. 

For  context  independent  knowledge  directed  adaptive  segmentation,  it  is  necessary 
to  have  either  well-defined  goals  or  a set  of  optimization  criteria  for  evaluating 
performance.  In  the  system  with  well-defined  goals,  performance  parameters  can  be  found 
that  measure  the  distance  fi'om  the  goal  at  any  stage  of  the  process.  These  parameters 
provide  a feedback  by  which  the  system  can  modify  its  strategy  for  segmentation. 
However,  the  goals  can  usually  be  defined  precisely  only  in  high-level  image  processing 
(e.g.,  shape,  color  distribution,  size,  etc.),  at  least  not  at  low-level  image  processing  (e.g., 
noise  removing,  edge  detection,  segmentation,  etc.).  To  establish  the  connectivity  between 
high-level  goal  and  low-level  processing.  Kohl  et  al  have  developed  a system.  They  devise 
an  intermediate-level  which  translates  high-level  goals  into  appropriate  low-level  process 
specifications  including  the  selection  of  algorithms,  parameter  setting,  etc.  In  this  way, 
high-level  interpretation  processes  gain  the  capability  of  refining  segmentation  data 
according  to  predefined  goals  and/or  current  hypotheses  regarding  the  content  of  the 
image,  and  thereby  improve  the  quality  of  the  overall  system.  Without  well-defined  goals, 
one  must  define  a set  of  measurement  parameters  for  evaluating  performance.  Thus 
knowledge  rules  can  direct  the  modification  of  segmentation  parameter  settings  based  on 
the  performance  measurements.  An  example  of  this  method  has  been  used  in  a rule  based 
image  segmentation  system  [Lev  85]. 

3.1.3.  Analytical  Adaptive  Scheme 

A more  analytical  adaptive  scheme  is  to  define  the  control  parameters  as  the 
function  of  pixel  values.  Thus  the  parameters  are  automatically  adjusted  during  image 
processing  according  to  the  changes  of  images.  For  example,  in  Hancock's  edge  detector 
[Han  91],  he  adaptively  estimates  the  hypothesis  thresholds  by  adopting  Bayesian 
framework  to  model  images.  In  this  way,  the  hypothesis  thresholds  used  in  Canny's  edge 
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detector  can  be  related  to  the  parameters  of  the  image  model  which  are  varied  with  image 
statistics.  This  kind  of  adaptive  scheme  is  easy  to  implement.  This  dissertation  adopts  this 
kind  of  adaptive  scheme. 

3.2  Segmentation  bv  Cost  Function  Optimization 
In  recent  years,  many  researchers  have  focused  their  research  on  algorithms  that 
segment  images  by  optimizing  some  cost  functions  (called  objective  functions  in  some 
papers).  The  cost  functions  measure  the  image  characteristics,  such  as  noise, 
discontinuities,  connectivities,  smoothness,  etc.  In  this  kind  of  segmentation  techniques,  it 
is  critical  to  properly  define  the  cost  function  and  the  process  of  obtaining  the  optimal 
solution  of  the  cost  function. 

3.2.1.  Cost/Objective  Function  Minimization  Approaches  to  Edge  Detection 

One  of  the  methods  of  cost  function  minimization  edge  detection  defines  cost 
function  as  measurement  of  edge  detection  quality.  The  basic  idea  used  in  the  cost 
function  minimization  edge  detection  is: 

1)  an  edge  detecting  algorithm  is  used  to  obtain  an  initial  edge  configuration; 

2)  a cost  fimction  is  introduced  that  measures  edge  quality,  such  as  continuity, 
smoothness,  thinness,  location,  etc.  to  form  a total  cost  function  C,  which  is  the 
weighted  linear  combination  of  all  edge  cost  functions; 

3)  C is  minimized  using  a suitable  optimization  technique. 

The  final  edge  configuration  is  one  with  minimum  C. 

Among  such  edge  detection  algorithms,  the  algorithm  due  to  Tan  et  al  [Tan  92] 
works  very  well  on  images  which  are  not  very  noisy.  The  algorithm  consists  of  two  stages, 
dissimilarity  enhancement  and  cost  fimction  minimization.  The  dissimilarity  enhancement 
detects  edge  candidates  by  calculating  the  degree  of  region  dissimilarity,  d(i,  j)  that  exists 
at  each  pixel  site  (i,  j)  and  assigning  pixels  as  edge  pixels,  s,  if  d(i,  j)  is  large  (0<d(i,  j)<l). 


39 


The  algorithm  defines  the  sum  of  a linear  combination  of  five  weighted  cost  factors  on  all 
pixels  as  the  cost  function,  or 

C(s)=ZZCk(s(i,j))  (3-1) 

i.j  k=  1 

The  problems  in  this  kind  of  edge  detection  algorithms  are:  (l)the  final  results  of 
this  algorithm  strongly  depend  on  the  initial  edge  configuration;  (2)  it  is  difficult  to  define 
an  effective  cost  function  and  edge  strategy;  and  (3)  reaching  a global  optimum  is  difficult. 

Another  kind  of  approach  for  cost  function  minimization  edge  detection  defines 
the  likelihood  on  edge  hypothesis  or  the  likelihood  ratio,  the  edge  hypothesis  to  non-edge 
hypothesis,  as  the  objective  function.  This  kind  of  approach  maximizes  the  function  at 
each  pixel  site  to  detect  edges.  Many  edge  detectors  based  on  Bayesian  decision  rule  use 
this  approach  [Fua  90]  [Jai  90]  [Tuc  91].  The  difficulty  of  this  kind  of  approach  is  to 
select  a proper  model  and  define  suitable  statistics  for  testing  the  edge  hypothesis  and/or 
non-edge  hypothesis.  For  example,  in  Jain  and  Nadabar's  algorithm  [Jai  90],  they  first  use 
the  Markov  random  field  (MRF)  to  model  the  edge  detection  problem.  Then  they  define  a 
3*6  pixel  neighborhood  for  calculating  the  edge  statistic  of  the  central  pixel  for  vertical 
edge  and  6*3  pbcel  neighborhood  for  horizontal  edge.  The  edge  statistic,  Es,  is  set  to  be 
equal  to  the  absolute  difference  of  average  intensity  value  between  9 left  side  pixels  and  9 
right  side  pixels  (or  9 up  side  pixels  and  9 down  side  pixels  for  the  horizontal  case).  The 
likelihood  distribution  of  non-edge  hypothesis  is  assumed  as  a t-distribution.  The 
likelihood  distribution  of  edge  h)q)othesis  is  set  to  be  a distribution  which  is  linear  within 
the  [0,  o(Es)]  interval  and  constant  in  (a(Es),  255]  where  o(Es)  is  the  standard  deviation 
of  Es.  The  likelihood  ratio  of  the  edge  hypothesis  to  non-edge  hypothesis  is  maximized  by 
the  Highest  Confidence  First  algorithm. 

A more  widely  used  cost  function  minimization  method  is  edge  detection  based  on 
the  cost  fimction  derived  from  discretization  of  the  functional  obtained  from  regularization 
principles  or  stochastic  estimate  techniques  used  in  Markov  random  field  (MRF).  Its  cost 
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function  is  now  some  energy  function,  membrane  energy  function  for  the  regularization 
model  and  Gibb's  energy  function  for  the  MRF  model  (in  fact,  these  two  energy  hinctions 
are  of  the  same  form  if  the  Gaussian  MRF  model— the  most  popular  model— is  used).  The 
variations  lie  in  the  difference  of  the  definition  of  each  term  in  the  function  and  the 
approaches  to  solve  the  optimization  problem.  The  theory  and  implementations  on  this 
kind  of  edge  detection  are  widely  available  in  the  literature  [Tor  86]  [Hua  87]  [Sim  88] 
[Dai  89]  [Li  90].  The  advantage  of  this  method  is  that  the  control  parameters  in  the 
process  can  be  made  adaptable.  (The  dissertation  will  describe  the  method  in  detail  in 
Chapters  5 and  6) 


3.2.2.  Region  Based  Segmentation  with  Cost  Function  Optimization 

MRF  model  region  based  segmentation  is  a cost  function  minimization  method  for 
region  based  segmentation.  The  cost  function  is  a Gibb's  energy  function  of  Gaussian 
MRF,  denoted  as  E.  It  is  defined  as: 


E=(Z%-^)+ZVc^„) 


m,n 


2*  <7^ 


(3-2) 


C€  C 


and 


-R  if  all  fn  in  C are  equal 

Vc(fn)={  (3-3) 

R otherwise 

where  d represents  input  image  data  set  with  M different  values,  f is  the  output  image  data 
set  with  N different  values  (N«M),  o is  the  standard  deviation  of  the  noise,  C is  a clique 
associated  with  a neighborhood  system  and  R is  a constant.  Segmentation  consists  of 
assigning  each  input  image  pixel  with  value  d^  to  a suitable  output  value  f^  which 
minimizes  the  cost  function,  E. 

The  first  term  of  E measures  the  error  caused  by  assigning  djjj  to  f^  when  djj^  is 
not  equal  to  f^.  The  second  term,  called  clique  potential  function,  measures  the  extra  cost 
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when  introducing  a discontinuity.  From  Eqs.  3-2  and  3-3,  we  can  see  that  the  MRF  based 
approach  is  in  fact  a clustering  method  with  spatial  constraints  since  it  is  the  same  as  the 
K-mean  cluster  algorithm  if  R=0.  By  adding  the  Vg(f)  term  to  the  K-mean  cluster 
algorithm  one  can  handle  the  noise  cases.  If  pixels  in  a neighborhood  are  noisy,  the  second 
term  plays  a major  role  in  minimizing  E,  that  is,  it  assigns  fjj  to  a d^,  which  tends  to 
smooth  the  neighborhood. 

Murray  [Mur  87],  Derin  [Der  87],  Wright  [Wri  89]  and  Chou  [Cho  89]  have  used 
the  principle  described  above  in  their  segmentation  algorithms  with  differences  in  the 
clique  structure  and  value,  the  optimization  approach,  and  additional  constraints.  The 
most  widely  used  optimization  approach  used  in  this  optimization  process  is  simulated 
annealing. 

Here,  the  parameter  R controls  the  final  segmented  results.  The  smaller  the  value 
of  R,  the  smaller  the  regions.  If  R is  too  small,  it  will  cause  incompact  segmented  regions. 
If  R is  too  large,  it  will  cause  oversmoothing  between  regions  and  therefore  cause  region 
deformations.  Another  controlling  parameter  is  the  number  of  desired  clusters  (objects), 
K.  If  K is  larger  than  the  actual  object  number,  false  segments  will  be  generated.  If  K is 
less  than  the  object  number,  some  objects  cannot  be  recognized. 

Another  kind  of  region  segmentation  with  cost  function  optimization  defines  the 
criterion  which  estimates  segmentation  quality  as  the  cost  function.  Monga  [Mon  87]  has 
developed  such  an  algorithm,  called  the  optimal  region  growing  segmentation  algorithm. 
He  defines  a criterion  function  which  is  a monotonic  and  symmetrical  function  of  M 
classical  criteria,  such  as  spatial  variance,  intensity  contrast,  mean-square-error,  etc.  The 
algorithm  first  initially  segments  images  into  a very  oversegmented  intermediate  result, 
then  merges  the  regions  according  to  the  optimal  value  of  the  criterion  function.  Since  the 
M criteria  are  decomposed  in  a hierarchical  manner,  the  optimization  procedure  is 
completed  by  finding  the  local  optimum  for  each  criterion  hierarchically  rather  than  by 
finding  the  global  optimum  for  all  M criteria.  Thus  the  optimization  is  simplified,  and  the 
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region  merging  is  also  hierarchically  carried  out.  The  segmentation  results  of  the  algorithm 
will  depend  on  the  selection  of  the  criteria  and  the  arrangement  of  the  hierarchy. 

3.3  Integrating  Region  and  Edge-Based  Segmentation 
Since  different  segmentation  schemes  have  their  own  distinct  advantages  and 
disadvantages,  and  the  results  from  different  schemes  are  usually  not  consistent,  many 
researchers  integrate  both  region  and  edge  information  in  the  segmentation  procedure  to 
retain  superiorities  and  overcome  shortcomings.  We  can  classify  these  approaches  as 
hybrid  segmentation  approaches.  There  are  two  schemes  for  integration  of  region  and 
edge  segmentation.  One  is  the  interactive  scheme,  another  is  the  result-modification 
scheme. 

3.3.1  Hybrid  Segmentation  with  Interactive  Scheme 

The  interactive  scheme  segments  images  by  either  utilizing  the  information 
obtained  from  edge  detection  as  additional  criterion  to  guide  the  region  based 
segmentation  procedure  in  forming  regions  or  as  constraints  to  prevent  forming  false 
regions,  or  employ  information  from  region  analysis  in  edge  detection. 

One  example  of  interactive  segmentation  algorithm  is  developed  by  Yu  and  Juha 
[Yu  91].  The  algorithm  combines  region  growing  with  derivative  edge  detection.  In  the 
algorithm,  the  uniformity  used  as  the  similarity  measurement  of  the  region  growing  is  gray 
level  uniformity,  and  the  edge  pixels  are  labeled  by  a Canny  detector.  The  algorithm 
iteratively  follows  the  following  steps: 

. Calculate  the  average  gray  level  of  pixel  fy  on  a window  centered  at  fy  with 
arbitrary  shape.  This  step  is  for  reducing  the  influence  of  noise. 

. If  the  gray  level  uniformity  of  fy  is  satisfied  and  fy  is  not  an  edge  pixel  in  the  edge 
model  image,  the  pixel  is  grown.  If  fy  is  an  edge  pixel,  a small  neighborhood  is 
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tested.  The  region  growing  procedure  stops  if  at  least  one  edge  pixel  is  found  among 
the  tested  neighbors,  otherwise  region  growing  continues. 

. If  gray  level  uniformity  of  is  not  satisfied  and  the  pixel  is  also  an  edge  pixel,  the 
region  Rowing  stops.  Otherwise  the  neighborhood  is  searched  for  edge  pixels.  The 
growing  procedure  stops  at  the  nearest  edge  pixel  which  is  located,  if  any  are  found. 
Otherwise  the  growth  stops  at  the  reference  pixel. 

The  algorithm  can  generate  more  accurate  region  boundaries  but  is  not  robust  to  noise 
because  both  region  growing  and  derivative  edge  detection  suffer  fi’om  noise. 

Wright  has  also  integrated  edge  information  in  his  MRF  region  based  segmenter 
[Wri  89].  He  introduced  a hidden  edge  variable,  Ij-  §,  into  the  original  cost  function  (Eq.  3- 
2).  If  s indicates  if  there  exists  an  edge  between  pixels,  r and  s.  If  so  If  § ^ 0,  else  If  g = 0. 
He  replaced  item  VQ(fjj)  (Eq.  3-3  ) by  V2(fn)  which  is  defined  as: 


V2(f„)={ 


Vc(fn)  if  lr,s=0 


0 


otherwise 


(3-4) 


In  this  way,  the  algorithm  does  not  introduce  an  extra  cost  function  for  edge  points,  and 
therefore  allows  the  existence  of  discontinuities  (edges). 

In  the  modular  feedback  image  segmentation  system  of  Anderson  et  al  [And  87], 
two-way  interactivity  is  used.  It  is  assumed  that  the  number  of  regions  is  known 
approximately  and  it  is  used  to  estimate  the  corresponding  parameters  of  an  edge 
detection  process.  Then  the  edge  detection  result  is  used  to  initialize  and  assist  a region 
growing  process  where  a local  similarity  threshold  T (which  is  used  to  judge  whether  two 
points  belong  to  the  same  region)  is  gradually  increased  until  the  expected  number  of 
regions  is  found. 


3 3 2 Integrated  Segmentation  Using  Result  Modification  Scheme 

Segmentation  using  result  modification  scheme  follows  three  steps:  (1)  generating 
initial  edge  based  segmentation  results,  (2)  generating  region  based  segmentation  results. 
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(3)  generating  final  segmentation  by  mapping  the  results  fi-om  (1)  and  (2).  The  simplest 
mapping  strategy  is  logical  AND  or  OR  operation.  However,  it  usually  cannot  yield  good 
results.  Research  on  mapping  strategy  resulted  in  many  algorithms  [Yok  89]  [Pav  90] 
[Had  90]. 

Pavlidis  and  Liow 's  mapping  strategy  [Pav  90]  employed  two  cost  functions:  one 
for  false  boundary  elimination,  denoted  as  F(e),  another  for  boundary  position 
modification,  denoted  as  Fjjj.  Both  functions  use  the  region  boundary  from  region  based 
segmentation  as  the  reference,  and  integrate  the  contrast  information  which  is  a variation 
of  the  image  gradient  along  the  boundary.  The  algorithm  generates  edge  information  by 
first  order  derivatives  and  regions  by  a quadtree  split-merge  algorithm.  F(e)  is  defined  as: 


F(e)  = 


+ P 


sum  of  contrasts  along  boundary  e 
length  of  boundary  e 


number  of  direction  change  along  boundary  e 
length  of  boundary  e 


(3-5) 


The  boundaries  with  "small"  F(e)  are  eliminated.  The  first  term  can  be  interpreted  as  the 
strength  of  the  boundary  seen  as  an  edge  segment.  The  second  term  penalizes  for  long 
straight  lines  which  most  often  correspond  to  artificial  boundaries  generated  by  the 
quadtree  split-merge  algorithm.  To  avoid  introducing  unacceptable  topologies,  two 
constraints,  region  constraint  and  boundary  constraint,  are  set.  Fjjj  is  defined  as: 


F„(i,j)=  f Z(|VI(i,j)|-  a.*r(i,j)-  r • l«>’(i.j)l)  (3-6) 

^ >.j 

The  boundary  point  is  modified  to  the  optimal  (i,  j)  which  maximizes  Fm(i,  j).  VI(i,  j)  is 
the  image  gradient,  which  corresponds  to  fidelity  to  data  since  it  is  maximum  when  the 
boundary  runs  along  the  points  of  maximum  contrast  or  gradient  magnitude.  x(i,  j)  is  the 
curvature,  which  favors  "smoothness"  and  avoids  sharp  turns  in  a contour.  j) 
corresponds  to  the  changes  in  direction  of  gradient  along  the  contour,  which  should  be 
small  since  large  (p'(*>  j)  means  noise  points.  The  algorithm  generates  accurate  region 
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boundaries  for  not  very  noisy  images  but  is  more  complicated  and  involves  many 
parameter  settings. 

All  the  advances  described  above  can  be  combined.  By  such  a combination,  one 
can  make  a segmentation  algorithm  with  adaptive,  optimum,  and  integrated  property,  and 
therefore,  make  maximal  use  of  various  advantages  in  different  advanced  segmentation 
algorithms  to  the  greatest  extent.  This  dissertation  develops  such  an  advanced 
combination  segmentation  approach. 


CHAPTER  4 

MAIN  PROBLEMS  ASSOCIATED  WITH  EXISTING  ALGORITHMS 


By  examining  various  segmentation  algorithms,  we  can  summarize  the  main 
problems  associated  with  them. 

1)  The  first  problem  is  the  threshold  or  parameter  setting.  All  of  the  algorithms  involve 

a threshold/parameter  setting  which  affects  the  segmentation  qualities  to  a great 
extent.  However,  setting  a suitable  threshold/parameter,  for  a practical  application, 
is  not  a trivial  problem,  especially  with  noisy  images.  Thresholding  itself  is  an  ill- 
posed  problem  and  the  skin  lesion  pictures  come  from  various  sources.  Therefore 
there  is  no  guarantee  to  find  the  right  threshold  for  the  set  of  all  input  images  and 
for  all  areas  of  the  images  needed  in  a practical  melanoma  identification  system. 
Fig.  4-1  shows  the  threshold  setting  effects  in  Sobel  edge  detectors.  A small 
threshold  will  give  rise  to  many  false  edges  like  the  result  shown  in  Fig.  4- 1(c)  and 
a large  threshold  will  cause  missing  of  true  edges  which  can  be  seen  in  Fig.  4- 1(d). 
Fig.  4-2  shows  the  parameter  setting  effect  in  a MRF  region  based  segmenter.  A 
small  parameter,  R,  gives  incompact  segmented  results  (see  Fig.  4-2(c)).  A large  R 
causes  oversmoothness  between  regions  and  therefore  causes  deformations  of  the 
regions  (see  Fig.  4-2(d)). 

2)  The  second  problem  is  noise  effects.  Any  noise  may  introduce  dissimilarities  into 
the  images.  Therefore,  schemes  using  differentiation  operations  (for  example  the 
edge  detectors)  will  enhance  the  noise  as  well  as  sharpen  edges,  and  the  schemes 
based  on  image  dissimilarities  (such  as  region  based  segmentation  algorithms)  will 
give  rise  to  false  results. 
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3)  The  third  problem  is  noise  suppression  effects.  To  handle  noisy  cases,  filters  with 
lowpass  property  (such  as  median  filters  and  Gaussian  filters)  are  used  to  remove 
noise  before  image  segmentation.  However,  these  filters  also  blur  images  which 
will  cause  shift  of  edge  positions  and  remove  true  edges  as  well  as  suppress  noise. 
Figs.  4-3  and  Fig.  4-4  show  the  effects  of  the  filter  size.  Fig.  4-3  is  the  results  of  a 
Sobel  edge  detector  combined  with  a median  filter,  and  Fig.  4-4  shows  the  results 
of  a LoG  edge  detector  in  which  a Gaussian  filter  is  used.  From  the  figures  we  can 
see  that  the  larger  the  filter  size  is,  the  stronger  the  ability  of  noise  removal  but  the 
more  the  missing  of  true  edges  and  the  less  accurate  the  edge  positions. 

4)  The  fourth  problem  is  the  inconsistency  problem.  Usually,  edge  based  segmentation 
results  and  region  based  segmentation  results  can  not  be  matched  very  well.  There 
are  three  kinds  of  inconsistencies  between  edge  based  and  region  based 
segmentation.  (1)  A region  boundary  is  not  an  edge  segment,  and  there  are  no 
edges  nearby.  (2)  A region  boundary  corresponds  to  an  edge  but  it  does  not 
coincide  with  the  edge.  (3)  There  exist  edges  with  no  boundaries  near  them.  Fig.  4- 
5 shows  the  inconsistencies. 
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Fig.  4-1  The  Effects  of  Threshold,  T,  in  a First  Order  Derivative  Edge  Detector 

(Sobel  edge  detector  with  a median  filter) 
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Fig.  4-2  Parameter,  R,  Effects  in  MRF  Region  Based  Segmentation  (4-level). 
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Fig.  4-3  The  Effects  of  Filter  Size,  W,  in  a First  Order  Derivative  Edge  Detector 
(Sobel  edge  detector  with  a median  filter) 
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Fig.  4-4  The  Filter  Size,  X,  Effects  of  LoG  Edge  Detector 
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Fig.  4-5  Inconsistency  Problem  between  Edge  Based  and  Region  Based  Segmentation 

(a)  Original  image  (256-level). 

(b)  Segmentation  result  from  region  based  approach. 

(c)  Edge  result  from  edge  based  approach. 

(d)  Mapping  image  of  edges  and  region  boundaries 

white  lines  — region  boundaries  not  matched  with  edges 
black  lines  — region  boundaries  matched  with  edges. 


PART  I 


LITERATURE  REVIEW 


Introduction 


To  solve  the  problem  described  in  Part  I,  a hybrid  segmentation  approach  was 
developed.  It  has  three  stages:  (l)PEDBOR  stage;  (2)LVDSO  quantization  stage;  and  (3) 
a mapping  stage. 

In  Stage  I,  a new  algorithm,  called  PEDBOR,  is  used  to  remove  noise  and  detect 
edges  simultaneously.  PEDBOR  is  the  abbreviation  of  Pbcel-Feature-Controlling  Edge 
Detection  Based  On  Regularization.  It  is  an  edge  detection  method  based  on  the 
regularization  model,  but  has  several  critical  improvements  compared  with  other 
regularization  processing  techniques.  The  improvements  are:  (1)  simple  optimization 
method,  (2)  a specialized  threshold  function,  (3)  classification  of  pixels  based  on  both 
magnitude  and  orientation  of  the  image  gradient,  (4)  the  adaptive  pixel-feature-controlling 
regularization  procedure.  These  empower  the  PEDBOR  algorithm  to  operate  without 
setting  parameters,  and  result  in  a well-posed  and  well-conditioned  process,  that  is,  robust 
to  noise.  By  using  PEDBOR  in  Stage  I,  both  noise  removed  images  and  edge  images  with 
high  quality  can  be  obtained.  They  provide  good  references  for  Stage  II  and  Stage  III. 

In  Stage  n,  a quantization  method  using  an  iterative  largest-variance-direction- 
split-optimization  (LVDSO)  in  the  histogram  domain  of  the  color  space  is  applied  to  the 
noise  removed  image.  Here  an  oversegmenting  parameter  is  used  to  quantize  the  noise 
removed  images  into  initially  segmented  images.  Oversegmenting  means  producing  more 
segmented  regions  than  there  are  objects  in  the  original  image.  Such  a parameter  setting 
has  two  advantages:  (1)  avoiding  the  difficulty  of  setting  an  exact  parameter,  and  (2) 
resolving  the  third  problem  of  inconsistencies  mentioned  in  Chapter  4,  namely  edges  with 
no  boundaries  near  them.  Of  course,  this  incurs  an  additional  computational  cost  due  to 
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the  many  false  regions  in  the  segmented  result.  These  false  regions  will  be  eliminated  in 
Stage  III. 

Stage  III  is  a mapping  procedure.  It  uses  the  edge  image  from  Stage  I as  the 
reference  of  true  regions,  and  uses  the  segmented  images  from  Stage  II  as  the  reference  of 
true  edges  to  remove  false  edges  and  false  regions.  The  final  segmentation  images  show 
uniformity,  compactness,  dissimilarity,  and  accurate  and  continuous  boundaries. 


CHAPTER  5 

REGULARIZATION  BASICS 


In  machine  vision  as  well  as  in  most  numerical  problems,  data  is  noisy.  Noise  in  the 
phototransduction  process  is  ultimately  unavoidable.  It  is  well  known  that  differentiation 
for  edge  detection  is  very  sensitive  to  noise  (including  noise  introduced  by  digitization). 
Even  a small  amount  of  noise  may  disrupt  the  differentiation  operation  needed  for  edge 
detection.  Thus  edge  detection  is  an  ill-posed  problem.  Well-posedness  and  numerical 
stability  of  the  differentiation  step  require  the  regularization  of  image  intensity  by  a 
regularizing  filter  operation  proceeding  differentiation.  The  regularization  provides  a 
general  framework  to  transfer  ill-posed  problems  in  the  early  vision  analysis  (e.g.,  edge 
detection,  surface  reconstruction,  etc.)  into  well-posed  problems  by  restricting  the  class  of 
admissible  solutions  using  constraints  such  as  smoothness. 

Regularization  of  the  ill-posed  problem  of  reconstructing  f fi’om  the  original  data  d, 
such  that  Af  = d can  be  described  as  selecting  suitable  norms  ||  • ||  (usually  quadratic)  and 
a stabilizing  functional  ||  Qf  ||.  There  are  three  main  methods  of  standard  regularization 
[Tor  86]: 

1)  Among  the  possible  f that  satisfy  ||  Qf  ||  < C],  find  f that  minimizes 

l|Af-d||  (5-1) 

2)  Among  the  possible  f that  satisfy  ||  Af-d  ||  < C2,  find  f that  minimizes 

IIQfll  (5-2) 

3)  Find  f that  minimizes 

II  Af-d  II  2 + ^IIQfll  2 (5.3) 

where  X,  is  a regularization  parameter,  and  Cj  and  C2  are  constants. 


56 


57 


The  first  method  consists  of  finding  the  function  f that  satisfies  the  constraint  ||Qf  || 
< C 1 , and  best  approximates  the  original  data.  The  second  method  computes  the  function  f 
that  is  sufficiently  close  to  the  data  d and  is  the  most  "regular",  or  the  most  "smoothing". 
In  the  third  method,  the  regularization  parameter,  X,  controls  the  compromise  between  the 
degree  of  regularization  of  the  solution,  that  is  the  smoothness  of  the  f,  and  its  closeness 
to  the  original  data.  If  the  regularization  is  viewed  as  a filter,  then  X controls  the  scale  of 
the  filter.  In  order  to  keep  edges  as  well  as  remove  noise  in  an  image,  the  third  method  is 
used.  The  simplest  forms  for  Q are  Q = d/dx  or  Q = d^/dx^  [Sch  64]  [Rei  67].  The  choice 
of  the  form  corresponds  to  a priori  constraint  of  smoothness  on  the  intensity  function  to 
handle  discontinuities  in  the  observation  or  in  the  reconstructed  attribute.  In  this  case,  Eq. 
5-3  can  be  written  as 


J [f(x)  - d(x)]  ^dx  + X J [f ' (x)]  ^dx 


(5-4) 


for  Q = d/dx. 


In  the  edge  detection  based  on  the  regularization,  edges  are  viewed  as 
discontinuities  with  certain  height.  Thus  the  edge  detection  problem  can  be  modeled  as  an 
optimization  problem  that  minimizes  the  membrane  energy  function 


E(f,d)=  JJ^(f-d)'dxdy  + X JJ„(ri+4)<lxdy 


(5-5) 


where  f^  = 5 f / d x and  fy  = d f / d y.  The  function  is  a variation  of  Eq.  5-4  in  two 
dimensions.  It  fits  a weak  membrane  (a  weak  membrane  is  a membrane  that  satisfies  weak 
continuity  constraints)  to  noisy  data.  The  edges  are  detected  by  marking  the  points  where 
discontinuities  in  the  membrane  height  occur.  This  regularization  model  can  remove  noise 
or  smooth  images,  and  keep  discontinuities  or  detect  edges  in  the  image.  The  trade-off 
between  smoothness  and  edge  detection  is  controlled  by  the  parameter  X. 

To  see  the  effect  of  X.,  we  can  view  the  regularization  operation  as  a regularizing 
filter.  If  the  edges  are  labeled  by  detecting  the  zero  crossing  of  f-d,  then  it  has  been  proven 
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that  all  discontinuities  (including  edges  and  noise)  with  height,  S,  satisfying  equation  5-6 
can  be  kept  in  f,  and  others  are  smoothed  [Lee  88  ]. 


A*  2v 


< S 


(5-6) 


where  v (>  0)  measures  the  slope  away  from  the  discontinuities.  From  this  equation,  we 
can  see  that  the  larger  the  X,  the  stronger  the  smoothing  ability  and  the  weaker  the  ability 
of  keeping  discontinuities  because  the  processed  image  data  becomes  more  regular  and 
retains  less  fidelity  of  f to  d.  Note  that  keeping  discontinuities  in  the  original  data  d is 
equivalent  to  choosing  f = d. 

To  simplify  the  problem,  divide  X into  two  parts,  or  let  X = L / p.  In  the  discrete 
case,  by  using  the  difference  method  and  letting  X = L / 3,  Eq.  5-5  can  be  written  as: 


E(f,d) 


(5-7) 


In  Eq.  5-7  L is  divided  into  two  components,  or  L = (L^,  L'')  ={(L|'j , Ln)}  to  control  the 

9 J 

spatial  smoothness  and  discontinuities  of  the  output  image  f in  both  x and  y directions. 
The  smoothness  varies  directly  with  the  magnitude  of  L.  3 controls  the  fidelity  of  f to  d. 
The  first  term  on  the  right  side  of  Eq.  5-7  measures  the  closeness  of  output  data  f to  the 
input  data  d.  The  second  and  third  terms  take  into  account  the  price  paid  whenever  a 
discontinuity  is  introduced,  and  thus  preventing  the  creation  of  discontinuities  everywhere. 

For  such  an  optimization  problem,  a suitable  optimization  method  should  be  used. 
In  standard  regularization  [Tik  77],  L is  chosen  such  that  the  membrane  energy  function  is 
a convex  function  and  therefore  a standard  calculus  optimization  method  is  used  in 
optimizing  E(f,  d).  In  this  way  the  computation  is  simplified.  However,  its  problem  is  the 
global  assumption  of  smoothness,  or  setting  Ljj,  by  disregarding  the  discontinuities  in  the 
scene  of  the  physical  characteristics  of  edges.  This  result  is  ill  suited  to  image  deblurring 
because  the  rapid  growth  of  E(f,  d)  as  (^J+i-  fy)  increases  inhibits  the  recovery  of  large 
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intensity  gradients  (thus  enforces  a strong  smoothness),  and  slow  growth  of  E(f,  d)  as 
(fiJ+1-  fy)  — > 0 promotes  excessive  smoothing.  Small  intensity  differences  (or  high  order 
analogs)  incur  a relatively  small  "penalty".  Thus  finally  the  transitions  become  gradual 
changes  and  the  solution  will  be  oversmoothed. 

To  bypass  this  problem,  nonstandard  regularization  models  [Ter  86][Li  90a] 
introduce  a piecewise  function  called  "line  process."  In  these  models,  L is  set  to  be  the 
line  process  function,  and  a new  cost  factor  measuring  the  line  process  itself  is  added  into 
Eq.  5-7.  Two  popular  line  process  functions  are  the  Boolean  function  and  the  sigmoid 
function  defined  as  follows.  The  Boolean  line  process  function  is 

1 if  |dx|=|fi,j-fi.j_i|<T 

Ly  = { . (5-8a) 

0 otherwise 


1 if  |dy|=|fy-fi_,j|<T 

Uj  = { (5-8b) 

0 otherwise 


The  sigmoid  line  process  function  is 


L”  = I - 1 / (1  + — I)) 


= 1 - 1 /(I  + 


(5-9a) 


(5-9b) 


Grraphs  of  the  line  process  functions  are  shown  in  Fig  5-1  and  Fig.  5-2.  Their  basic  idea  is 
that  when  the  directional  gradient  magnitude  of  the  pixel,  |dx|,  is  larger  than  a threshold  T, 
the  Ljj  is  set  to  be  "Off',  otherwise  to  be  "On".  The  "Off'  state  means  Lij  = 0 for 
Boolean  line  process  function,  or  Ljj  = small  value  for  sigmoid  function.  The  "On"  state 
corresponds  Ljj  = 1 for  Boolean  line  process  function,  or  Ljj  = large  value  for  sigmoid 
function.  When  the  discontinuities  have  large  intensity  gradient  magnitudes,  Ljj  is  small 
and  the  total  penalty  is  still  small.  Thus  the  discontinuities  with  high  steps  larger  than  T are 
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kept  and  therefore  the  oversmoothing  is  relieved.  The  values  of  Ly  vary  inversely  with 
discontinuity  height.  The  value  Ly  = 1 leads  to  the  strongest  smoothness,  and  Lj;  = 0 
prevents  any  smoothing  operations.  In  this  way,  the  line  process  function  controls  the 
smoothness  and  discontinuities  of  the  solution.  The  main  problems  associated  with  such 
line  process  function  settings  are; 

(1)  The  cost  function  is  not  convex,  therefore  other  optimization  methods,  such  as 
simulated  annealing,  direct,  or  relaxation  method,  are  needed  to  solve  the 
optimization  problem.  However,  the  simulated  annealing  method  is 
computationally  expensive,  and  the  other  two  methods  converge  only  when  the 
penalty  term  is  large,  which  also  generates  an  oversmoothed  solution. 

(2)  The  line  process  function  setting  regards  the  discontinuities  only  in  the  scene  of 
discontinuity  magnitude,  which  will  cause  the  following  problems: 

. All  discontinuities  which  have  gradient  magnitudes  larger  than  a threshold,  T,  will 
be  kept  no  matter  whether  they  are  noise  or  not,  and  all  pixels  with  gradient 
magnitudes  less  than  T will  be  smoothed  no  matter  whether  they  are  true  edges 
or  not. 

. Setting  T is  a difficult  problem  for  practical  applications. 
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Fig.  5-1  Boolean  Line  Process  Function 
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Fig.  5-2  Sigmoid  Line  Process  Function 


CHAPTER  6 
PEDBOR  ALGORITHM 


A new  algorithm  has  been  developed  to  solve  the  problems  in  traditional 
regularization  procedures.  The  new  algorithm  is  called  PEDBOR,  an  abbreviation  of  Pixel- 
Feature-Controlling  Edge  Detection  Based  On  Regularization.  It  uses  the  same 
regularization  model  defined  by  Eq.  5-5  with  a line  process  fimction,  but  has  several 
critical  improvements  compared  with  other  regularization  processing  techniques.  These 
improvements  are: 

(1)  Use  of  a non-interacting  line  process  function  in  the  optimization.  This  not  only 
makes  optimization  simple,  but  also  allows  employing  additional  knowledge  about 
the  physical  nature  of  edges  to  control  the  regularization  procedure. 

(2)  Both  magnitude  and  orientation  information  of  an  image  gradient  are  used  in 
classifying  image  pixels  as  edge  pixels  (E-pixels),  homogeneous  region  pixels  (HR- 
pixels),  and  noise  pixels  (N-pixels).  This  information  is  used  to  set  line  process 
functions. 

(3)  For  the  different  types  of  pixels,  different  line  process  functions  are  set  based  on 
the  desired  line  process  function  properties.  This  prevents  the  oversmoothing  of 
edges  and  undersuppression  of  noise.  The  other  parameter  that  controls  the  fidelity 
of  processed  images  to  the  original  images  is  also  set  according  to  the  pixel  type. 
Therefore,  the  whole  regularization  course  is  adaptivelly  controlled  by  pixel 
features. 

(4)  In  the  computation  of  the  line  process  fijnction  and  pixel  classification,  a data 
driven  threshold  function  is  used  for  all  images.  This  makes  threshold  setting 
adaptable  and  avoids  the  difficulty  of  setting  a threshold. 
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PEDBOR  detects  edges  during  iterative  regularization  by  finding  the 
discontinuities  L = { Ljj  j ((i,  j)  e pixelgjgg)  which  separate  regions  with  homogeneous 
properties.  Each  iteration  consists  of  two  main  phases:  a classification  phase  and  a noise 
removal  phase.  In  Phase  I,  pixels  are  grouped  into  three  categories:  edge  pixels  (E-pixels), 
homogeneous  region  pixels  (HR-pixels)  and  noise  pixels  (N-pixels).  The  classification 
criterion  uses  information  about  the  magnitude  of  the  gradient  and  the  orientation 
distribution  of  the  gradient  in  the  neighborhood  of  each  pixel.  The  threshold  used  in 
classification  is  a preset  function  of  image  intensity.  It  has  low  enough  values  so  that 
almost  all  true  edges  can  be  detected  for  most  images,  as  well  as  a number  of  extraneous 
ones.  Later  these  extra  edges  are  pruned  by  examining  their  gradient  orientations.  In  Phase 
II,  a regularization  process  is  applied  to  the  pixels  to  remove  noise.  During  regularization, 

information  obtained  from  the  classification  phase  is  used  to  activate  line  processes  and  to 
set  fidelity-controlling  parameters. 

6. 1 . Optimal  Update  Rule  for  Minimizing  the  Cost  Function  E (f.  dl 

As  mentioned  above  when  line  process  is  introduced  into  the  cost  function  E(f,d), 

the  standard  calculus  optimization  methods  are  not  suitable  to  use  since  the  function  is  not 

convex.  However,  if  we  use  the  line  process  function  in  a ndn-interactive  way,  we  can  still 

utilize  the  standard  calculus  method.  Non-interactive  way  involves  two  aspects:  (1)  terms 

do  not  interact  with  each  other,  (i.e.,  there  is  no  term  for  "organizing"  the  boundaries 

added  to  the  E(f,  d)  );  and  (2)  they  do  not  interact  with  the  E(f,d)  (i.e.,  find  the  minimum 
of  E(f,  d)  by  iteratively  finding  individual  minE(f,d)  for  each  given  B,  Lh  and  L,^:)-  Under 

this  assumption,  the  optimal  update  rule  for  minimizing  the  function  E (f,  d)  can  be 
obtained  by  the  following  analysis. 

The  membrane  energy  function  E is  the  functional  of  f,  given  d.  Now  introduce  a 
new  variable  t into  f,  or  f = { f (x,  y,  t)  }.  This  new  variable  may  be  thought  of  as  'time' 
indicating  the  evolution  of  the  Energy  function  as  the  data  is  refined,  hence  as  a function 
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of 'time'.  Thus  the  E is  a function  of  the  parameter  t,  given  f and  d.  To  find  the  minimum 
E,  we  require  that  the  energy  E should  not  increase  along  the  trajectory  f (x,  y,  t)  as  the 
system  evolves  with  time,  or  dE/dt  ^ 0.  A simple  dynamic  system  with  characteristic  dE/dt 
< 0 is  determined  by  the  differential  equation 


^f(x,y.t) 


(6-1) 


where  p > 0 is  a normalization  quantity.  Thus,  a djmamic  updating  rule  for  the  continuous 

minimization  problem  can  be  obtained  by  setting  the  first  derivative  of  the  cost  Sanction, 
E(f,d)  with  respect  to  f j ; to  zero  for  given  P,  Ln  and  Lh,  or 


<?E(f,L)  ^ 


^fi.j 


(fu“du)^'''— (fij~di,j)^+... 


+L|*o(fu  fi,o)  +Lo,i(fi,i  fo.i) 


} 


= 0 


or 


^(fiJ~di,j)'  + [L|:j_i(fi,j-  fi.j-,)-  L5j(fi.i.i-  fij)] 

+ [Lr_y(fij-  fi-ij)-  UjCfi+ij-  fi.j)]  = 0 

By  rearranging  this  equation,  the  optimal  update  rule  for  minimizing  the  function  E (f,  L) 


is  as  follows: 


f I j * (Ly  f lj+ 1 j_  1 f Ij- 1 Ly  1,  j i,j  f i-  y + ^i,j  dij)  / A 


(6-2a) 


and 


A=(L!;i+L;'_._+L,>L,^,,i+A,i) 


(6-2b) 


From  the  last  formula  we  can  see  that  fj j in  the  (t+l)f^  iteration  is  in  fact  equal  to 
the  weighted  average  of  its  four  neighbors  plus  the  effect  of  input  data,  dj;.  The  weights 
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are  just  equal  to  the  values  of  Lj;.  Fig.  6-1  shows  the  relationship  of  Eq.  6-2a.  The  larger 
the  Lj j , the  stronger  the  smoothness.  If  j = 0,  no  smoothness  will  be  allowed,  and  at 
the  same  time  a discontinuity/edge  is  signaled  at  site  (i,  j).  Now  we  can  determine  L and  3 
based  on  the  knowledge  of  the  physical  properties  of  different  data  types  without  affecting 
the  generalization  of  the  problem.  That  is,  the  regularization  procedure  can  be  adaptivelly 
controlled  by  additional  knowledge  about  the  physical  nature  of  the  edges  to  get  better 
results. 


Fig.  6-1  The  4-connectivity  Neighborhood  Used  in  the  Update  Rule. 

6.2.  JND  Threshold  Function 

An  additional  improvement  is  the  JND  threshold  function.  JND  means  "just 
noticeable  difference".  It  is  well  known  that  the  human  eye  possesses  different  capabilities 
for  luminance  discrimination  under  different  background  conditions.  The  visual  threshold 
is  lower  in  the  medium  gray  region  and  much  higher  in  the  very  bright  and  dark  regions. 
One  research  result  [Wan  91]  shows  that  the  JND  of  luminance  depends  on  the 
background  luminance  intensity,  and  puts  forth  a JND  threshold  function  defined  by  the 
following  formula: 
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2*(-0.85  Ijj  + 40)  for  0^  Ijj  < 40 

T(i,j)  = *{  12  for  40<  Ijj  < 100 

2*(0.09  ly  - 3)  for  100<  ly  < 255. 

The  numeric  values  in  the  above  expression  depend  on  monitor  settings  and  vary 
depending  on  the  hardware  employed,  but  they  may  be  adapted  to  a standardized  setup. 
In  this  dissertation  a monitor  setting  was  used  that  allowed  the  maximum  intensity  just  at 
the  maximum  light-level  of  the  display  CRT.  By  examining  many  standard  gray  level  and 
color  images,  a modified  JND  threshold  hmction  was  developed.  It  is  more  suitable  to 
PEDBOR  algorithm: 

-0.5  ly  + 38  for  0<  ly  < 64 

T(i,j)  = 'i  6 for64<Iy<160  (6-3) 

0.05  ly  - 2 for  160<  ly  < 255 

where  ly  is  the  representative  intensity  value  of  a pixel  at  site  (i,  j).  For  noiseless  images, 
ly  is  equal  to  fj j.  For  noisy  images,  ly  is  the  average  intensity  of  pixels  in  (i,  j)'s 
neighborhood,  or 


Ii,j  Q S 2fi+k,j+l 


(6-4) 


The  difference  between  original  function  and  modified  function  can  be  seen  easily 
from  Fig.  6-2.  Such  a threshold  function  is  data  driven.  Since  the  modified  function  has 
low  values,  it  can  be  used  for  most  pictures  without  missing  of  true  edges.  Of  course,  it 
also  gives  rise  to  many  false  edges.  To  eliminate  the  false  edges,  the  pixel-feature- 
controlling regularization  is  needed. 


6.3.  Classifying  Pixels 

To  suppress  noise  and  preserve  true  edges,  identifying  them  is  the  key  point  in  the 
algorithm.  In  the  previous  nonstandard  regularization  approach,  pixels  are  just  classified 
as  edge  pixels  and  non-edge  pixels  by  using  only  magnitude  information  of  the  image 
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Fig.  6-2  JND  Threshold  Function 
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gradient.  Unfortunately,  magnitude  information  of  the  image  gradient  is  not  enough  to 
identify  the  true  edge  pixels  and  noise  pixels  since  noise  pixels  also  have  a large  gradient 
magnitude.  Hence,  PEDBOR  utilizes  both  magnitude  and  orientation  information  of  the 
image  gradient  and  edge  structure  information  to  classify  image  pixels. 

Gradient  information  needed  in  classifying  pixels  can  be  calculated  in  the  following 
manner;  the  gradients  of  pixel  (i,  j)  in  horizontal  direction  and  in  vertical  direction  are 


j)  = (fi,j+i“  fi,j-i)/  2 (6-5a) 

dy(i,  j)  = (fi+i,j  - fi-uj)/  2 (6-5b) 

respectively;  the  gradient  magnitude  and  orientation  of  pixel  (i,  j)  are 

I dx  (i,  j)  I,  I dy  (i,  j)  I (6-6a) 

GO(i,  j)  = QUA[tan"  *(dx(i,  j)/  dy(i,  j)]  (6-6b) 


respectively.  In  the  above,  QUA[.]  means  quantizing  the  continue  "arctan"  operation  value 
into  one  of  the  equally  divided  orientation  zone  values.  For  example,  an  8— orientation 
zone  has  8 different  values:  0,  45,  90,  135,  180,  225,  270,  315  and  360  degrees  (see  Fig. 
6-3). 


Fig.  6-3  8-orientation  Zone  of  GO 

For  an  n*n  pixel  region,  we  can  identify  it  as  a constant  intensity,  a noisy,  or  an 
edge-containing  one  by  analyzing  the  distribution  of  GO(i,  j)  ((i,  j)  the  n * n 
neighborhood).  Regions  with  a unique  orientation  are  constant  intensity  ones.  Regions 
with  two  or  a few  more  strong  orientations  may  contain  several  line  fragments  in  different 
directions,  such  as  a curve,  a comer  or  line  crossing.  Regions  with  equal  orientation  scores 
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in  many  directions  are  noisy  regions.  Fig.  6-4  shows  the  GO  distributions  of  different 
regions. 
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Fig.  6-4  GO  Distributions  of  Different  Regions. 

(a)  An  edge-containing  region. 

(b)  A constant  intensity  region. 

(c)  A noisy  region. 

Besides  using  gradient  information  of  images,  we  also  use  the  physical  structure 
information  of  edges.  Considering  that  edges  are  the  boundary  of  different  regions,  they 
must  have  certain  structures.  In  other  words,  some  edge  structures  are  forbidden.  In  our 
application,  the  edge  structures  shown  in  Fig.  6-5  and  their  rotations  in  any  direction  are 
forbidden.  Edges  with  length  less  than  3 pixels  are  also  forbidden. 


Fig.  6-5  Forbidden  Edge  Structures 
x--edge  pixel.  0— non-edge  pixel,  blank— edge  or  non-edge  pixel. 


Based  on  the  analysis  above,  we  can  obtain  the  following  classification  rule: 
pixel  (i,  j)  is  an  E-pixel  if  |dx(i,  j)|  > T(  Ijj)  or  |dy(i,  j)|  > T(  ly),  GO(i,  j)'s  ((i,  j)  e 
n*n  neighborhood)  have  less  than  4 directions  (for  small  noise  image  less  than  5)  and 
(i,  j)  is  not  a forbidden  edge  pixel; 


• pixel  (i,  j)  is  an  HR-pixel  if  |dx(i,  j)|  < T(  I jj)  and  |dy(i,  j)|  < T(  I y); 

. pixel  (i,  j)  is  a noise  pixel  (N-pixel)  if  it  is  neither  an  E-pixel  nor  an  HR-pixel. 
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6.4.  Setting  the  Line  Process  Function 

We  have  seen  the  problems  caused  by  standard  regularization  and  nonstandard 
regularization  with  Boolean  and  sigmoid  line  process  functions.  These  line  process 
function  settings  do  not  consider  the  characteristics  of  different  pixels.  That  is,  they  do  not 
satisfy  the  properties  of  a desired  line  process  function.  A desired  line  process  function 
should  have  the  following  properties: 

. For  noise  pixels,  the  larger  the  noise,  the  larger  the  values  of  the  line  process 
functions  and  therefore  the  stronger  the  smoothness; 

. For  those  intensity  regions  which  are  more  sensitive  to  noise,  the  function  should 
have  high  values  (note  that  regions  with  high  ability  of  luminance  discrimination  are 
also  more  sensitive  to  noise); 

. For  true  edge  pixels,  the  function  value  should  be  zero  to  inhibit  any  smoothness. 
Besides,  the  selection  of  the  line  process  function  must  guarantee  the  stability  of  the 
iteration  process. 

Based  on  these  properties,  a set  of  line  process  functions  is  established  in 
PEDBOR  according  to  the  pixel  types.  For  noise  pixels,  the  line  process  function  is 
defined  as: 


ij  ^ 


: if  |dx|>T(I:  ;) 

(l+exp(-|dx|/T(Ii  j)))  ‘’J 


1- 


1 


(l+exp(T(lij)-|dx|)) 


otherwise 


(6-7a) 


if  |dy|>T(I;  :) 

V (l+exp(-|dy|/T(li  j)))  ’J 

L*  • “ 1 


(l+exp(T(Ii  j)-|dy|)) 


otherwise 


(6-7b) 
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For  edge  pixels,  the  line  process  function  is  set  as: 


1 

(l+exp(T(lij)-|dx|)) 


if  |dx|>T(Ii  j) 
otherwise 


(6-8a) 


0 

1 ! 

(l+exp(T(ijj)-|dy|)) 


if  |dy|>T(Iij) 
Otherwise 


(6-8b) 


For  homogeneous  region  pixels,  the  line  process  fiinction  has  the  following  form; 


1 

(l+exp(T(lij)-|dx|)) 


(6-9a) 


1 

(l+exp(T(lij)-|dy|)) 


(6-9b) 


The  corresponding  curves  of  these  line  processes  are  shown  in  Fig.  6-6.  In  the 
figure,  (a)  and  (b)  correspond  to  the  line  process  function  for  noise  pixels,  (c)  to  one  for 
edge  pixels,  and  (d)  to  one  for  homogeneous  pixels.  The  curve  in  Fig.  6-6  (a)  is  with 
respect  to  the  magnitude  of  the  pixel  gradient.  We  can  see  that  the  larger  the  magnitude, 
that  is  the  stronger  the  noise,  the  larger  the  function  value  and  therefore  the  stronger  the 
smoothness.  Fig.  6-6  (b)  is  with  respect  to  the  intensity  level.  From  this  curve,  we  can  see 
that  the  region  which  is  most  sensitive  to  noise,  or  intensity  levels  between  64  and  160, 
has  the  highest  function  values.  The  other  two  line  process  functions  satisfy  the  desired 
properties  also. 

As  mentioned  before,  the  regularization  parameter,  X = L / P,  automatically 
controls  the  scale  of  the  filter  if  the  regularization  is  viewed  as  a filter.  Thus,  such  a setting 
for  line  process  functions  is  equivalent  to  form  a scale  space  adaptivelly  adjusted  by  pixel 
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(a)  Line  Process  Function  for  Noisy  Pixels  with  Respect  to  the  Magnitude 
of  the  Pixel  Giradient. 


Fig.  6-6  Line  Process  Functions 
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(b)  Line  Process  Function  for  Noisy  Pixels  with  Respect  to  the  Intensity  Level. 


Fig.  6-6  Continued 
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(c)  Line  Process  Function  for  Edge  Pixels. 


Fig.  6-6  Continued 
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(d)Line  Process  Function  for  Noisy  Pixels 


Fig.  6-6  Continued 
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characteristics.  Is  it  a useful  scale  space?  The  analysis  in  section  6-6  will  answer  this 
question. 

6.5.  Implementation  of  PEDBOR 

The  PEDBOR  can  be  implemented  by  iteratively  following  the  steps  below. 

Step  1 . Initialize  f = d. 

Step  2.  Calculate  T(Ijj)  for  each  pixel  using  Eqs  6-3  and  6-4. 

Step  3 . Classify  each  pixel  using  Eqs.  6-5  and  6-6,  and  the  classification  rule. 

Step  4.  Calculate  line  process  values  using  Eqs.  6-7  to  6-9  and  set  the  parameter  P j ; as 
follows: 


0.01 


0.001 


for  HR-pixel  and  E-pixel 
for  N-pixels. 


Step  5.  Calculate  new  fy  using  Eq.  6-2. 

Step  6.  Repeat  step  2 to  step  5 until  ~ N*E-pixd  I < e ( N*E_pixei  is  the  number 

of  E-pixels  in  the  t th  iteration  ) or  iteration  times  are  larger  than  a predetermined 
number. 

Step  7.  Scan  in  line  process  domain  and  label  a pixel  at  site  (i,  j)  as  an  edge  pbcel  if  |dx(i,  j)| 
> T(fy)  or  |dy(ij)l  > T(fy). 


6 6 Analvas  of  PROBOR 

In  this  section  we  present  properties  of  PEDBOR,  such  as  stability,  smoothing  and 
edge  keeping  ability,  and  iterative  behavior.  For  simplicity,  the  1-D  signal  case  is  used, 
and  the  dependencies  on  the  input  data  are  ignored,  that  is  let  P = 0. 


78 


6.6. 1,  Stability  of  PEDBOR 

The  stability  problem  concerns  the  unbounded  growth  or  the  controlled  decay  or 
boundness  of  the  exact  solution  of  the  finite-difference  equations.  The  essential  criterion 
defining  stability  is  that  the  numerical  process  should  limit  the  amplification  of  all  effects 
due  to  initial  conditions. 

The  iterative  process  of  PEDBOR  represented  by  Eq.  (6-2)  can  be  written  in 


matrix  form  as  follows: 


ft+l-At  ft  +d 


(6- 10a) 


where  A,  called  iteration  matrix,  is  an  N*N  matrix  of  known  elements.  It  is  defined  by 


At  = 


•••  ••• 


0 


Lx-2 


Lx-1 


Lx-2  + Lx-1 


•••  ##• 


0 


Lx-2  + Lx-1 
Lx-1 


L x-1  + L 


X 


••• 


0 


Lx-1  + L 


»#•  ••• 


0 


••• 


(6-1 Ob) 


N is  the  number  of  data  points.  In  order  for  the  process  to  be  stable,  for  some  norm.  At 
must  satisfy 


l|At  II  < 1 


for  V t 


so  that  a general  property  of  the  definition  of  a norm  follows  that: 


l|An-lAn-2  ...  A^Aq  ||  < ||An.i||  ||An.2l|...  ||Ai||  ||Aol|<  1 


for  n iterations. 


(6-11) 


(6-12) 


The  infinity  norm  of  a matrix  is  simply  the  maximum  sum  of  the  moduli  of  the 
elements  of  the  matrix.  Since  the  values  of  the  line  process  function,  L^,  are  always 
positive,  the  infinity  norm  of  our  iteration  matrix  is  unity  because 


l|AtlL  = 


Lx— 1 


+ 


= 1 


Lx— 1 Lx  Lx-1  L] 


(6-13) 


In  addition,  the  largest  moduli  of  the  eigenvalues  of  the  square  matrix  cannot  exceed  its 
infinity  norm,  i.e., 

p (A)  < IIAtIloo  (6-14) 

which  means  that  the  2-norm  of  A^  is  also  bounded  by  unity.  Therefore,  the  adaptive 
iteration  process  is  stable,  since  jjA^  ||  < 1 is  always  guaranteed  during  the  iterative 
process. 


6.6.2.  The  Ability  of  Smoothing  and  Edge  Keeping. 

The  smoothness  of  the  image  and  the  detection  of  edges  in  the  results  of 

PEDBOR  are  controlled  by  line  process  fianctions.  In  order  to  observe  the  effects  of  our 

line  process  fimctions  or  the  abilities  of  edge  smoothness  and  edge  keeping  of  PEDBOR, 

d cif  df* 

we  evaluate  the  term  — (| — |)  which  represents  the  change  of  gradient  or  contrast  I — I 

dt  dx  dx 

d df 

along  the  iteration  measure  dimension  (t).  If  — (| — |)  > 0,  the  gradient  magnitude  will 

dt  dx 

increase  as  the  iteration  progresses,  achieving  a discontinuity  sharpening  effect.  If 
d df  d df 

t(It“I)  “ the  discontinuity  step  is  kept.  If — (| — |)  < 0,  then  gradient  magnitude  will 
dt  dx  dt  dx 

be  smaller  and  smaller  as  iterations  progress  and  the  discontinuities  will  be  smoothed 
eventually.  Therefore,  the  following  relation  should  be  satisfied; 


® for  noise  or  homogeneous  pixels 

dt  dx  t „ . . . 


> 0 for  edge  pkels 


(6-15) 


By  exchanging  the  order  of  the  differentiation,  Eq.  (6-15)  can  be  rewritten  as: 
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We  can  get  the  solution  of  Eq.  (6-16)  by  first  finding  dfidt.  We  find  dfidt  by  the  follovvdng 
analysis. 


Recalling  Eq.  (6-2),  PEDBOR  can  be  viewed  as  an  iterative  weighted  averaging 
process.  In  1-D  case,  Eq.  (6-2)  can  be  reformulated  as: 

f t+1  (x)  = C t (x)  * ft  (x-1)  + C t (x+1)  * ft  (x+1)  (6-17a) 

with 


Ct  (x)  + Ct  (x+1)  =1  and  0 < Ct  (x),  Ct  (x+1)  < 1 
to  guarantee  the  stability  of  the  iterative  process.  In  the  equation. 


C‘(x)  = 


V(x) 

L*(x)  + L*(x  + 1)  ’ 


C'(x-M)=  , 

L (x)  + L (x  + 1) 


(6- 17b) 


Rewrite  Eq.  (6-1 7a)  as  follows: 

f t+1  (x)  -ft  (x)  = C t (x)  * ft  (x-1)  + C t (x+1)  * ft  (x+1)  - ft  (x) 

= Ct  (x+1)  * [ft  (x+1)  - ft  (x)]  - C t (x)  *[f t (x+1)  - ft  (x)] 

_ L*  (x  + 1)  * [f^  (x  + 1)  - f*  (x)]  - L*  (x)  *[f*  (x  + 1)  - f*  (x)] 

L‘  (x  + l)+L‘  (x) 

that  is 


<7t 


1 

L‘  (x+l)+L‘  (x) 


V(LVf) 


(6-18) 


where  V = — corresponding  to  the  differentiation  in  the  spatial  domain.  Here  Lt(x+x)  + 
Lt(x)  > 0.  Eq.  (6-18)  can  be  represented  as: 


t^f(^»t)  ^ 1 /AfL tx  tl A(^it) 

dt  L(x  + l,t)+L(x,t)  dx  ’ dx 

Note  that  we  will  denote  df(x)/dx  as  f^(x)  in  the  following. 

Plugging  Eq.  (6-19)  into  Eq.(6-16),  we  can  get: 


(6-19) 
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A(^)  = A A ^ A( \ A. 

dtMx^  dx  A(x  + l,t)+L(x,t)  dx 


[L(x,t)fx(x,t)]} 


_ Lxx(x,t)f^(x,t)  + 2Lx(x,t)fxx(x,t)  + L(x,t)fxxj(x,t) 

L(x  + l,t)  + L(x,t) 

+ [Lx(x,t)fx(x,t)  + L(x,t)fxx(x,t)]*  A[ 


1 


(6-20) 


dx  L(x  + l,t)+L(x,t) 


] 


Usually,  we  model  a discontinuity  in  a real  image  as  a step  function  blurred  by  the 
lowpass  property  of  equipment.  Fig  6-7  shows  the  profiles  of  such  discontinuities.  For 
profile  (a),  at  the  location  of  the  discontinuity,  fx  > 0 and  fxxx  0.  For  figure  (b),  fx  < 
0,  and  fxxx  At  the  point  of  inflection,  fxx=  0 since  the  inflection  point  corresponds 
to  the  point  with  maximum  slope.  Under  the  settings  of  the  line  process  function  of 
PEDBOR,  Lx  = 0 because  fxx=  0. 

The  following  relations  could  be  obtained  by  using  Eq.  (6-20)  and  corresponding 
line  process  functions.  If  the  discontinuity  is  caused  by  noise,  which  has  L(x,  t)  of  Eq. 
(6-7), 


d(|df|>w_  L(x,t)|f,oo,(x,t)|  M I Ux,t)|fx(x,t)|  , |fx(x,t)| 

dt^'dx'^  L(x  + l,t)  + L(x,t)  ^ T j U<U 


(6-21) 


where  T is  the  threshold.  If  the  discontinuity  corresponds  to  a homogeneous  pixel,  for 
which  L(x,t)  is  represented  by  Eq.  (6-9), 


L(^»0|fxxx(x,t)|  Ifxl 

dt  'dx'  L(x  + l,t)+L(x,t) l + exp(T-lfxi) 
If  the  discontinuity  corresponds  to  an  edge  pixel,  then 


) 


dt^'dx'^  ^ 


L(x, t)[fxxx(x,t)|  _ jv 

L(x  + l,t)+  L(x,t)  2 
0 


for|fx(x,t)|=  T 


Note  that  in  PEDBOR,  ~ 6. 


(6-22) 


for|fx(x,t)|>  T 


(6-23) 
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Fig.  6-7  Profiles  of  a Discontinuity. 
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From  Eq.  (6-21),  (6-22)  and  (6-23),  we  can  obtain  the  following  conclusions.  (1) 

d df 

For  noise  pixels,  — (|— |)  < 0 and  also  the  larger  the  noise  the  larger  the  L(x,t),  and 

dt  dx 

d df 

therefore  the  larger  the  magnitude  of  — (|— |),  which  causes  a stronger  and  quicker 

dt  dx 

d df 

smoothness.  (2)  For  homogeneous  pixel,  whether  — (| — |)  > 0 or  not  will  depend  on  the 

dt  dx 


factor  1- 


l+exp(T-|fJ) 


. Fig  6-8  shows  the  curve  of  the  factor.  From  it  we  can  see 


d df 

that  — (I— I)  < 0 when  | fx  | < T.  (3)  All  edges  with  | fx  | = T are  sharpened  because 
dt  dx 

“(I'pD^O,  and  other  edges  with  | fx  | > T are  kept  since  — (|— 1)=  0.  These  results 
dt  dx  dt  dx 

agree  with  the  properties  of  the  desired  line  process  functions  described  in  the  beginning 

of  section  6.4. 

Note  that  the  sigmoid  line  process  function  used  by  some  previous  regularization 

d df 

algorithms  has  the  — (| — |)  defined  as; 

dt  dx 


d f.  df  . 2*f^(xt) 

— ( — )=  C*  L(x,t)f^(x,t)[l 

dt  'dx'  V , / XXXV  , A l + exp(T-f2(x,t)) 


] 


(6-24) 


where  C is  a constant. 

The  curve  of  the  sigmoid  line  process  function  is  shown  in  Fig.  6-9.  It  is  evident 


d df 

that  the  — (|— |)  is  not  larger  than  zero  for  those  discontinuities  with  the  Ify  I's  which  are 
dt  dx  ^ 


not  large  enough  to  make  L(x,t)  s 0 even  though  the  discontinuities  correspond  to  true 
edges.  On  the  other  hand,  when  the  discontinuities  are  very  large  so  that  the  L(x,t)  e 0, 
forbidding  smoothness,  they  will  be  kept  even  though  they  are  caused  by  noise.  For 
Boolean  line  process  function. 
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Fig.  6-8  The  Curve  of  1 - 


Ifxl 

l + exp(T-|fx|) 
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Fig.  6-9  The  Curve  of  1 - 


2*f^(x,t) 
l + exp(T-  f^(x,t)) 
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d,|df|.  f lfxxx(x>t)l 


if|f,^x,t)|>T 

otherwise 


(6-25) 


which  will  keep  all  discontinuities  with  |fx(x)  | > T,  and  smooth  other  ones. 


6.6.3.  Iterative  Behavior 

Theoretically,  the  termination  criterion  of  the  implementation  of  PEDBOR  should 

be  the  convergence,  that  is,  no  pixel  value  changes  in  the  final  output  will  occur  in  a new 

s 

iteration.  In  this  case,  the  resulting  image  consists  of  piecewise  constant  regions. 
However,  this  needs  extremely  large  number  of  iterations.  It  is  important  to  realize  that 
there  are  two  different  operations  affecting  the  images  as  the  iteration  proceeds;  one  is  the 
detecting  of  edges  that  will  survive,  the  other  is  the  smoothing  of  regions,  i.e.,  noise 
removal.  For  edge  detection,  50  iterations  have  been  found  to  be  enough  for  the  types  of 
images  that  are  subject  of  this  work.  For  noise  removal,  more  iterations  lead  to  better 
results.  A comprise  iteration  number  is  100. 

PEDBOR  will  generate  two  output  images.  One  is  an  edge  image  consisting  of 
edge  segments  which  are  thick  and  not  continuous  but  have  accurate  positions  and  fewer 
missing  and  false  elements.  The  other  is  a noise  removed  image  consisting  of  regions  with 
small  spatial  variation.  Instead  of  using  a great  many  iterations  to  get  the  constant  region 
image  and  using  thinning  algorithms  and  connecting  algorithms  to  get  thin  and  continuous 
edges,  our  hybrid  segmentation  approach  generates  constant  regions  in  Stage  II 
(quantizing  noise  removed  images  to  regions,)  and  forms  thin  and  continuous  boundaries 
in  Stage  III  (mapping  the  edge  image  with  the  region  image  result  fi'om  the  second  stage 
of  this  hybrid  approach.) 
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6.7  Practical  Considerations  in  the  Implementation  of  the  PEDBOR 

6.1. 1 ■ Use  of  a Lookup  Table  for  Speed  Up 

In  PEDBOR,  to  calculate  the  gradient  orientation  of  a pixel,  the  arctan  operation  is 
applied  on  each  pixel  in  each  iteration.  The  calculation  of  the  arctan  is  very  time 
consuming.  For  example,  on  an  IBM  PS2-80/386,  each  arctan  calculation  needs  205  ps. 
Thus  an  image  of  512*512  pixels  will  take  90  minutes  for  just  the  arctan  calculations  in 
the  1 00  iterations.  A lookup  table  can  be  used  to  speed  up  this  calculation.  Recall  Eq.  (6- 
6b)  for  computation  of  gradient  orientation;  we  can  see  that  for  a 256-level  image,  the 
lookup  table  has  512  * 512  = 262,144  entries  ( 512  for  x-direction  and  512  for  y- 
direction).  Thus  263  kb  extra  storage  is  needed  for  a software  implementation.  When  the 
algorithm  is  implemented  under  Windows  environment,  the  lookup  table  can  be  preloaded 
into  an  addressed  memory  and  all  entries  are  directly  accessible.  In  this  case,  the  time  of 
searching  for  the  entries  becomes  neglibile  and  the  time  for  arctan  calculations  is  reduced 
to  the  time  of  loading  the  lookup  table.  Under  memory  limited  environments,  although  the 
lookup  table  can  be  stored  in  several  separated  data  segments,  the  lookup  table  is  usually 
not  preloaded  into  memory  but  on  the  hard  disk  with  a file-open  status.  In  this  case, 
various  fast  search  strategies  can  be  used  for  entry  search.  To  reduce  the  storage  and 
speed  up  the  search  operation,  the  lookup  table  with  262144  entries  can  be  modified  to 
one  with  fewer  entries  by  combining  triples  with  the  same  orientation  values.  After 
eliminating  the  redundancy  in  both  x-direction  and  y-direction,  the  number  of  entries  can 
be  reduced  to  1529,  and  the  storage  size  can  be  reduced  from  263  kb  to  30  kb.  Table  6-1 
in  Appendix  A gives  the  modified  lookup  table. 

6.7.2.  Implement  PEDBOR  Block  bv  Block 

When  image  processing  algorithms  are  implemented  on  the  PC  and  under  non- 
Windows  environment,  an  image  must  be  divided  into  blocks  and  processed  block  by 
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block  because  of  the  limitation  of  memory  size  (usually,  each  data  segment  cannot  exceed 
64  kb,  and  total  data  segment  is  limited  to  640  kb,  dependent  on  the  memory  model  being 
used).  In  this  case,  block  effects,  such  as  artificial  edges  between  blocks,  the  missing  of 
those  edges  located  at  four  block  boundaries,  intensity  non-consistency  of  adjacent  blocks, 
etc.,  may  occur  in  the  output  images.  To  prevent  these  block  effects,  block  overlap  is 
necessary.  The  overlap  size  must  be  suitable  since  block  sizes  not  large  enough  to  overlap 
can't  eliminate  block  effects  and  too  large  an  overlap  size  increases  the  redundancy  of 
operations.  Usually,  a minimum  overlap  size  is  adopted.  For  the  PEDBOR  algorithm,  the 
minimum  overlap  size  is  3 on  each  side  (see  Fig.  6-10),  1 for  gradient  operation,  1 for 
edge  structure  mapping,  and  1 for  regularization  operation.  For  efficiency,  the  block  size 
should  be  as  large  as  possible,  dependent  on  the  memory  size. 

6.7.3.  A Consideration  for  Parallel  Implementation  of  the  Algorithm. 

The  algorithm  can  be  implemented  in  parallel  in  task  based  or  block  based  parallel 
scheme.  The  former  assigns  different  processors  to  different  tasks,  such  as  edge 
information  calculation,  edge  structure  mapping,  and  regularization.  The  latter  assigns 
different  processors  for  each  subimage  block.  During  parallel  implementation, 
synchronizing  must  be  set  in  each  iteration  cycle  for  the  block  based  scheme  as  well  as  for 
the  task  based  scheme.  In  many  algorithms,  no  synchronization  is  necessary  during 
iterations.  The  operations  are  applied  on  every  block  in  the  whole  iteration  without 
interactive  information  propagation  between  adjacent  blocks.  By  setting  synchronization  in 
each  iteration,  blocks  are  merged  after  each  iteration  and  redivided  for  each  new  iteration, 
and  therefore  the  modified  information  of  each  block  in  the  last  iteration  will  be 
propagated  by  the  overlap  pixels.  In  this  way,  intensity  inconsistency  between  adjacent 
blocks  can  be  prevented. 


overlap  overlap  overlaq? 
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Fig.  6-10  Block  Dividing  Strategy 


CHAPTER  7 

LVDSO  QUANTIZATION 


PEDBOR  can  generate  both  edge  images  and  noise  removed  images  with  high 
quality.  However,  the  noise  removed  images  are  still  intensity  images  with  256  levels 
rather  than  well-segmented  images.  Thus  it  is  necessary  to  use  region  based  segmentation 
algorithms  to  segment  the  images  into  object  regions.  In  our  hybrid  segmentation 
approach,  an  iterative  largest-variance-direction-split-optimization  algorithm  (LVDSO)  is 
used  to  quantize  noise  removed  images  into  initially  segmented  images.  The  reasons  for 
employing  the  quantization  technique  are  that: 

1 ) In  region  segmentation,  assigning  input  image  data  with  a large  number  of  different 
values  to  output  data  with  a small  number  of  different  values  is  equivalent  to 
requantizing  the  input  data. 

2)  The  procedure  has  only  one  parameter  that  needs  to  be  set.  The  parameter  is  the 
number  of  clusters,  K,  in  an  image. 

3)  The  processing  can  be  carried  out  in  the  histogram  domain  rather  than  in  the  spatial 
domain.  This  allows  faster  processing  since  the  histogram  has  at  most  3 * 256 
different  entries  for  a 24  bit/per  pixel  color  image. 

LVDSO  belongs  to  global  histogram  thresholding  segmentation  techniques.  But 
here  the  optimum  thresholds  are  found  by  minimizing  the  average  sum-of-squared-errors 
which  are  called  quantization  errors  and  equal  to  the  difference  between  the  segmented 
image  and  the  input  image.  Such  a problem  can  be  modelled  as  follows. 

Consider  the  input  data  with  N distinct  values  denoted  by  a set  {X^}  (n  =1,  2,  ..., 
N).  The  objective  of  the  quantization  is  to  find  K (K<<N)  representative  levels  (Rk)  (k 
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=1,  2,  K),  called  centers  of  cluster  such  that  the  average  sum-of-squared-errors 

defined  by 

E=Z  ||X„-R^|P  •P(X„)  (7-1) 

n=l 

is  minimum,  where  P (X^)  is  the  probability  of  Xjj  occurring.  If  the  cluster  center  is 
chosen  as  the  statistical  mean  of  the  corresponding  cluster,  that  is, 

Rk  = Uk=  ZXn*P(Xn)  (7-2) 

Xn  € Qj^ 

then  the  Eq.  (7-1)  becomes 

E=  I ll^n-Uj^F  *P(^n>=  ^ (^-3) 

x„  e Qk  k = 1 

In  this  equation  is  the  variance  of  cluster  That  means  we  can  assign  the  input 

pixel  with  level  Xj|  to  a cluster  which  minimizes  the  sum  of  the  variance  of  each 
cluster. 

This  is  another  optimization  problem.  Instead  of  using  regular  optimization 
methods,  such  as  standard  calculus,  simulated  annealing,  direct,  or  relaxation  methods,  we 
find  the  solution  of  the  optimization  problem  by  using  the  iterative  largest-variance- 
direction-split-optimization  procedure.  The  procedure  is  described  below. 

For  a fiill  color  image  with  256  different  levels  for  each  primary  color,  quantizing 
must  be  carried  out  in  3 -bands  of  Red,  Green  and  Blue,  which  is  a 3 -dimensional 
quantization  problem.  Assume  that 

Input  data  set:  Xj-  (I),  Xg  (I),  X5  (I)  ( I = 0,1,2,. ..,255) 

Output  data  set:  (kj-),  Ug  (kg),  Ub  (kb),  (k^  + kg  + kj-  = K) 

The  computational  procedure  follows  the  steps  presented  below. 

Step  1.  Initialize  k=l  and  calculate  the  histograms  of  each  band: 

Hr={  hr  (I)  },  Hg={  hg  (I)  },  Hb={  hb  (I)  } (1=0,  1,  2, ... , 255) 
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hz(I)  = S y ye  (0,  255),  z = r,  g,  b (7-4) 

y=I 

and  the  probabilities: 

p(Xz(I))  = ^^^,  z = r,  g,b.  (7-5) 

where  L is  the  number  of  pixels  in  an  image. 

Step  2.  Calculate  the  variances  and  means  of  each  band  of  data  using  Eq.  (7-2)  and 
(7-3). 

Step  3.  Partition  one  band  with  largest  variance  into  two  regions.  The  partition  point 
Topt  is  located  at  the  point  that  minimizes  E(T)  defined  by 


E(T)=  Z||X(I)-Uki(T)|P*P(X(I)) 

x(i)  e 

+ EI|X(I)-Uk2(T)lP*P(X(I)) 

x(i)  e Q^2 


(7-6) 


Step  4.  Calculate  the  variances  and  means  of  nifi(Topt)  and  0^2  (Topt)  set  k = 
k+1. 


Step  5.  Check  if  k = K,  if  so  stop  the  iteration,  otherwise  go  to  Step  3. 

Step  6.  Assign  input  data  into  the  closest  Topt 

In  the  procedure,  the  parameter,  K,  is  predetermined.  Our  hybrid  segmentation 
approach  quantizes  the  image  with  an  oversegmenting  parameter,  or  a larger  K than  the 
expected  number  of  regions  in  the  image,  to  avoid  picking  an  accurate  K and  to  ensure 
that  all  true  object  edges  are  preserved  in  the  segmentation.  We  calculate  K by  the 
following  formula; 


K = Np^(R)  + Np^(G)  + Np^(B)  + 1 (7-7) 

where  Npe^kW^  Npeak(G)  and  Npeajf(B)  are  the  number  of  peaks  in  the  R,  G and  B 
histograms  respectively,  and  Noverlap  *s  the  number  of  the  same  peak  positions  in  the  R, 
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G and  B histograms.  Fig.  7-1  gives  an  example  of  calculating  K.  Note  that  the  number  of 
actual  clusters  in  the  segmented  image  is  equal  to  kf  kg  * kj-  rather  than  K. 

By  using  the  LVDSO,  an  oversegmented  image  can  be  obtained.  We  call  the 
oversegmented  image  the  initially  segmented  image.  In  the  initially  segmented  image,  there 
are  still  some  false  regions  due  to  the  oversegmentation  and  the  inconsistency  of  the 
histogram  thresholding  segmentation  scheme  itself  These  false  regions  will  be  eliminated 
in  Stage  III. 


✓ 


F)  band 


B band 


^peak  C^)  ^peak  (^)  ^peak  (®)  ^overlap 
K = 2 +2  +1  - 1 +1  =5 


Fig.  7-1  The  Example  of  the  Calculation  of  K 


CHAPTER  8 

MAPPING  EDGE  IMAGES  WITH  INITIAL  SEGMENTATION 

As  mentioned  above,  the  results  of  the  edge  based  segmentation  and  region  based 
segmentation  are  not  consistent.  There  are  three  kinds  of  inconsistencies  between  edge 
based  and  region  based  segmentations.  (1)  A region  boundary  is  not  an  edge  segment,  and 
there  are  no  edges  nearby.  (2)  A region  boundary  corresponds  to  an  edge  but  it  does  not 
coincide  with  the  edge.  (3)  There  exist  edges  with  no  boundaries  near  them.  See  Fig.  4-5. 
The  main  reasons  causing  the  inconsistencies  are  the  inconsistency  existing  between  the 
different  segmentation  schemes  themselves  as  well  as  noise  effects  and  unsuitable 
parameter  settings.  Many  edge  detectors  adopt  the  area  operation  scheme,  while  many 
region  based  segmenters  use  the  point  operation  scheme.  For  example,  derivative  edge 
detectors  detect  region  boundaries  by  detecting  the  magnitude  of  the  difference  between 
two  adjacent  points,  while  region  based  segmenters  form  region  boundaries  by  assigning 
each  pixel  to  a class  based  on  the  similarity  distance  between  the  pixel  and  the  center  of 
that  class.  Usually,  it  is  difficult  to  say  which  segmentation  result,  edge  based  one  or 
region  based  one,  is  better.  A more  reasonable  way  is  adopting  an  integration  scheme, 
which  combines  the  information  of  both  edge  based  and  region  based  schemes,  to  get  a 
more  consistent  result.  On  the  other  hand,  noise  and  unsuitable  parameter  setting  also 
cause  some  false  edges  and  false  regions.  An  example  is  the  third  type  of  inconsistency 
mentioned  above,  error  caused  by  an  oversegmentation  parameter,  or  larger  than 
necessary  K set  in  Stage  II  of  our  developed  approach.  However,  decreasing  such  a 
parameter  setting  increases  the  first  type  of  inconsistency. 

In  order  to  get  a high  quality  segmentation  result,  our  hybrid  segmentation 
approach  uses  a mapping  procedure.  It  utilizes  the  edges  in  the  edge  image  as  the 
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boundary  reference  of  the  true  regions  and  the  boundaries  of  initially  segmented  images  as 
the  true  edge  reference  to  remove  false  edges  and  false  regions.  The  mapping  procedure 
includes  two  phases. 

• Map  the  edge  image  with  the  boundaries  of  the  region  image  to  form  true  region 
boundaries. 

• Merge  those  regions  between  which  no  true  boundaries  exist. 

The  basic  idea  of  the  mapping  procedure  is  described  as  follows. 

Consider  the  boundary  between  an  arbitrary  pair  of  adjacent  regions  R1  and  R2. 
Assume  that  there  are  Njj  points  along  the  boundary.  These  N5  points  are  called  boundary 
candidates.  For  each  candidate,  verify  whether  there  is  a corresponding  edge  point  in  the 
edge  image;  if  so  mark  it  as  a matched  point.  Let  Ne  be  the  number  of  matched  points.  If 
the  ratio  Ne/N^  is  larger  than  some  value  (in  our  approach,  the  value  is  between  0.6  and 
0.7  ),  it  is  likely  that  the  boundary  is  a true  boundary.  As  a result,  the  final  region 
boundaries  consist  of  all  true  boundaries.  If  no  boundary  exists  between  two  regions,  the 
regions  are  merged. 

In  Stage  II,  a histogram  quantization  algorithm  is  used  to  obtain  initially  segmented 
images.  Thus  region  patches  are  labeled  by  their  cluster  centers  in  the  histogram  domain. 
This  means  that  two  patches  with  different  spatial  location  but  similar  properties  may  have 
the  same  labels  since  the  spatial  information  cannot  be  contained  in  the  image  histogram. 
Therefore  it  is  necessary  to  relabel  region  patches  by  combining  spatial  information  to  give 
each  patch  a unique  label  before  performing  the  mapping  procedure.  The  relabeling  course 
is  completed  by  a grouping  procedure.  The  grouping  procedure  employs  8-connectivity  as 
spatial  measurement,  alternatively  growing  and  tracking  back  to  group  pixels  with  the 
same  cluster  center  labels  and  8-connectivity  together. 

After  the  relabeled  image  is  obtained,  the  mapping  procedure  is  applied  to  it.  The 
mapping  procedure  consists  of  the  following  steps. 
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Step  1.  From  top  to  bottom  and  left  to  right,  for  the  region,  Rj,  calculate  those 
mapping  ratios  which  exist  between  Rj  and  its  adjacent  regions. 

Step  2.  Check  all  these  mapping  ratios.  If  there  are  any  regions  with  the  mapping  ratio 
smaller  than  the  predetermined  value  (say  0.7),  then  pick  out  one  of  them  which 
has  the  minimum  mapping  ratio,  and  implement  Step  3.  Otherwise  go  to  Step  4. 
Step  3 . Between  Rj  and  the  selected  region,  merge  the  smaller  region  into  the  larger  one, 
then  go  to  Step  1 . 

Step  4.  Check  whether  or  not  the  mapping  procedure  has  been  applied  to  all  regions.  If  so, 
terminate  the  procedure.  Otherwise  go  to  Step  1 . 

Fig.  8-1  shows  an  example  of  the  merge  strategy  (assume  the  mapping  ratio 
threshold  is  set  to  0.7).  Fig.  8-1  (a)  is  the  initial  region  labels.  In  this  region  configuration, 
the  mapping  ratios  between  region  E and  A,  region  D and  C,  and  region  B and  C are  all 
less  than  0.7  (see  Fig.  8-1  (b)).  Since  the  mapping  procedure  is  applied  to  each  region  in 
the  fashion  of  top-to-bottom  and  left-to-right,  region  A is  first  merged  into  region  E,  and 
region  C is  then  merged  into  region  D.  After  these  mergings,  the  region  labels  become  as 
shown  in  Fig.  8-1  (c).  The  corresponding  mapping  ratio  graph  is  shown  in  Fig.  8-1  (d).  At 
this  time,  only  the  mapping  ratio  between  region  D and  B is  less  than  0.7.  Therefore 
region  B is  merged  into  region  D.  The  final  region  labels  and  corresponding  mapping  ratio 
graph  are  shown  in  Fig.  8- 1 (e)  and  (f)  respectively. 
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(b)  Mapping  ratio  graph  of  (a) 


Fig.  8-1  Merging  Strategy  Used  in  the  Mapping  Procedure 
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(c)  Region  labels  after  merging  region  E and  A,  and  region  D and  C 


(d)  Mapping  ratio  graph  of  (c) 


Fig.  8-1  Continued 
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(f)  Mapping  ratio  graph  of  (e) 


Fig.  8-1  Continued 


CHAPTER  9 

EXPERIMENTAL  RESULTS 


In  this  chapter,  many  expriement  results  are  presented.  Since  the  PEDBOR 
algorithm  is  the  key  of  the  developed  approach,  the  chapter  first  evaluates  PEDBOR  in 
section  1 by  comparing  it  with  several  other  tranditional  algorithms  to  show  the  quality  of 
performance  of  PEDBOR.  All  these  algorithms  have  been  tested  on  both  s)mthetic  images 
and  real  images,  including  melanoma  photographs.  Quantitative  evaluation  is  made  on 
three  synthetic  images  with  different  signal-to-noise  ratios.  The  visual  evaluation  was  done 
on  both  synthetic  images  and  real  images.  Section  2 shows  the  visual  evaluation  results  of 
applying  the  hybrid  segmentation  approach  to  images  of  malignant  melanoma. 

9. 1 Evaluation  of  PEDBOR 


911  Quantitative  Evaluation  of  PEDBOR  Algorithm 

To  evaluate  the  PEDBOR  algorithm,  quantitative  evaluation  was  made  on  three 
synthetic  images  with  different  signal-to-noise  ratios,  SNR  = 1,  3,  5 and  7 dB.  The  first 
image  is  a 4-level  image  with  background  intensity  200,  and  object  intensities  of  150,  100, 
and  50  (Fig.  9- 1(a)).  The  second  image  contains  two  bars  with  3-  and  4-pixel  widths  ( Fig. 
9-2(a)  ).  The  third  image  is  a checkerboard  ( Fig.  9-3(a)  ).  Both  bar  image  and 
checkerboard  image  have  background  intensity  150  and  foreground  intensity  100. 
Gaussian  noises  with  standard  deviation  of  10,  16,  25  and  39  are  added  to  the  images  to 
obtain  noisy  images  with  SNR  values  of  7,  5,  3 and  1.  Figs.  9-1  to  9-3  show  the  three 
images  and  corresponding  noisy  images. 
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In  the  quantitative  evaluation,  the  SNR  for  edge  detection  is  defined  as: 

SNR  = 101og-  (9-1) 

a 

where  h is  the  edge  height  and  a is  standard  deviation  of  the  noise.  In  all  three  images,  h's 
are  all  equal  to  50. 

Three  quantities:  Pf,  Pjjj  and  are  used  as  performance  measures.  Pf  and  Pm 
evaluate  edge  detection  results.  Pf  is  called  the  false  alarm  probability,  which  is  equal  to 
the  conditional  error  probability  of  assigning  an  edge  pixel  given  a non-edge  pixel.  P^  is 
called  the  missed  detection  probability,  which  is  equal  to  the  conditional  error  probability 
of  assigning  a non-edge  pbcel  given  a true  edge  pbcel.  is  the  measurement  evaluating 
noise  removed  image.  is  the  average  variance  of  the  image,  which  measures  the 
smoothness  of  non-edge  regions  of  an  image.  The  smaller  the  Pf,  Pm  and  Sm,  the  better 
the  results.  Note  that  the  false  alarm  pixels  are  not  only  really  false  object  edges  but 
include  thickened  edges  and  false  object  edges  as  well.  Similarly,  missing  detection  pixels 
are  not  just  real  missing  object  edges  but  include  thinned  edges  and  missing  object  edges. 
So  two  other  measures,  false  object  edges  and  missing  object  edges,  are  used  to  indicate  if 
there  are  false  object  edges  introduced  and  if  any  object  edges  are  missing. 

For  comparison,  two  other  algorithms  are  applied  to  the  same  images.  First  is  the 
Sobel  edge  detector  of  threshold  T with  a median  filter  of  filter  size  W,  denoted  as 
MedSob.  Second  is  the  nonstandard  regularization  algorithm  with  sigmoid  line  process 
function  of  parameter  T 1 followed  by  a Sobel  edge  detector  of  threshold  T2,  denoted  as 
SIGMR.  The  results  from  both  Sobel  edge  detectors  and  regularization  approaches  are  the 
best  ones  of  the  experimental  results  with  different  parameters.  The  "best"  means  fewest 
false  and  missing  objects. 

Tables  9-1  to  9-3  give  the  quantitative  evaluation  results.  In  the  tables,  some 
detectors  have  two  results,  (1)  and  (2),  under  the  same  SNR,  where,  (l)'s  correspond  to 
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the  results  of  minimum  missing  object  edge  cases  and  (2)'s  to  the  results  of  minimum  false 
object  edge  cases. 

9.1.2  Visual  Evaluation 

The  visual  evaluation  was  done  on  both  synthetic  images  and  real  images.  Figs.  9-4 
to  9-9  show  the  results  of  three  synthetic  images.  Fig.  9-10  shows  the  evaluation  of  the 
checkerboard  image  surface,  from  which  we  can  more  directly  compare  the  ability  of  noise 
removal  and  edge  keeping  of  the  algorithms. 

The  real  image  is  a standard  gray  level  image,  "Lena".  Corresponding  noisy  images 
of  "Lena"  are  generated  by  adding  Gaussian  white  noise  with  standard  deviation  of  20  and 
30.  Fig.  9-11  shows  the  original  "Lena"  image  and  its  noisy  counterparts.  Four 
algorithms;  PEDBOR,  MedSob,  SIGMR  and  LoG,  are  used  on  these  images.  The 
parameters  for  both  MedSob  and  SIGMR  have  the  same  meanings  as  above.  The  LoG 
edge  detector  has  the  controlling  parameter  X.  Figs.  9-12  and  9-13  show  a set  of 
corresponding  results. 

From  the  experiments,  we  can  see  that  PEDBOR  has  the  best  performance.  It  is 
much  more  robust  to  noise  than  other  algorithms.  It  gives  rise  to  fewest  false  edges  and 
missing  edges  and  gives  more  accurate  edge  positions  in  edge  images.  Also  its  smoothed 
images  have  the  least  blurring  on  edges  and  the  most  smoothing  in  homogenous  areas. 
PEDBOR  is  a well-conditioned  as  well  as  a well-posed  algorithm,  that  is,  its  solution  is 
robust  against  noise.  By  using  the  PEDBOR,  a high  quality  noise  removed  image  and  an 
edge  image  can  be  obtained.  They  provide  a good  references  for  Stage  II  and  Stage  ITT 

9.2  Application  to  the  Computer  Diagnostic  System  of  Malignant  Melanoma 

Figs.  9-14  to  9-15  show  several  experimental  results  of  our  hybrid  segmentation 
approach  for  images  of  malignant  melanoma.  In  the  figures,  (a)s  are  original  images,  (b)s 
and  (c)s  are  noise  removed  images  and  edge  images  generated  by  PEDBOR,  (d)s  are 
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quantization  results  of  (a)s,  (e)s  are  quantization  results  of  (b)s,  or  initially  segmented 
images,  and  (f)s  are  final  segmented  images.  Comparing  several  segmented  results  fi'om 
different  stages  shown  in  pictures  (d)s,  (e)s  and  (f)s  respectively,  we  can  see  that  the 
segmentation  of  the  original  image  is  not  uniform  and  compact,  and  the  boundaries  are  not 
continuous  because  of  the  effects  of  noise  and  the  roughness  of  the  skin  lesions.  By  using 
PEDBOR,  noise  in  the  images  are  reduced  greatly.  However,  there  are  still  many  false 
regions  in  the  initial  segmentation  results  shown  in  the  (e)  pictures  due  to  the  improper 
quantization  parameter  and  the  inconsistency  of  the  algorithm  itself.  After  the  mapping 
stage,  the  final  segmented  images  have  the  best  results.  That  is,  they  have  the  fewest  false 
regions,  their  regions  are  uniform,  compact  and  show  dissimilarity,  and  the  region 
boundaries  are  continuous  and  thin. 
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Table  1 . Quantitative  Evaluation  Results  of  4-level  Image 


SN 

R 

detector  type 

Pm 

Pf 

Sm 

false 

objects 

missing 

objects 

PEDBOR 

0.02114 

0.00443 

2.6153 

No 

No 

7db 

MedSob  (W=3,  T=30) 

0.01126 

0.01405 

13.8342 

No 

No 

SIGMR  (Tl=40,  T2=10) 

0.03738 

0.03078 

8.9567 

No 

No 

PEDBOR 

0.04711 

0.00988 

5.4109 

No 

No 

5 db 

MedSob  ( W=7,  T=20) 

0.22632 

0.02474 

12.671 

No 

No 

SIGMR  (Tl=50,  T2=10) 

0.09012 

0.04375 

15.842 

Some 

No 

PEDBOR 

0.08039 

0.02489 

20.102 

No 

No 

3db 

MedSob  (1)  (W=5,  T=20) 

0.19406 

0.06500 

38.983 

Some 

No 

MedSob  (2)  (W=7,  T=20) 

0.22376 

0.04205 

30.605 

No 

some 

SIGMR(T1=50,  T2=15) 

0.24270 

0.05228 

38.207 

Many 

No 

PEDBOR 

0.11009 

0.03572 

29.907 

Fewer 

No 

1 db 

MedSob(l)  (W=7,  T=20) 

0.22427 

0.08113 

70.819 

Many 

No 

MedSob  (2)  (W=7,  T=30) 

0.48643 

0.03493 

70.819 

Some 

Many 

SIGMR  (T1=50,T2=15) 

0.15361 

0.23077 

317.156 

Many 

Some 
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Table  2.  Quantitative  Evaluation  Results  of  Bar  Image 


SN 

R 

detector  type 

Pm 

Pf 

Sm 

false 

objects 

missing 

objects 

PEDBOR 

0.00605 

0.00043 

2.2954 

No 

No 

7db 

MedSob  (W=3,  T=30) 

0.01010 

0.00026 

13.112 

No 

No 

SIGMR  (Tl=40,  T2=10) 

0.38911 

0.00026 

5.9277 

No 

Some 

PEDBOR 

0.01815 

0.00284 

5.7683 

No 

No 

5db 

MedSob  (W=3,  T=30) 

0.05645 

0.00302 

35.662 

No 

No 

SIGMR  (T1=50,T2=10) 

0.48185 

0.00379 

8.2973 

Few 

Many 

PEDBOR 

0.05040 

0.00646 

13.472 

Few 

Few 

3 db 

MedSob  (1)  (W=3,  T=30) 

0.17339 

0.05731 

92.436 

Many 

Some 

MedSob  (2)  (W=5,  T=30) 

0.67742 

0.00310 

34.819 

Few 

Many 

SIGMR  (T  1=50,  T2=10) 

0.46169 

0.53430 

39.491 

Some 

Many 

PEDBOR 

0.20161 

0.02404 

31.470 

Few 

Few 

1 db 

MedSob  (1)  (W=5,  T=20) 

0.42943 

0.17511 

90.672 

Many 

Some 

MedSob  (2)  (W=7,  T=20) 

0.89718 

0.03817 

48.292 

Some 

Many 

SIGMR  (T1=50,T2=10) 

0.36694 

0.28421 

367.493 

Many 

Many 
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Table  3 Quantitative  Evaluation  Results  of  Checkerboard  Image 


SNR 

(db) 

detector  type 

Pm 

Pf 

Sm 

false 

objects 

missing 

objects 

PEDBOR 

0.00332 

0.00027 

2.27263 

No 

No 

7 

MedSob  (W=3,  T=20) 

0.00664 

0.00027 

13.0030 

No 

No 

SIGMR  (Tl=30,  T2=15) 

0.02434 

0.00518 

8.71332 

Few 

Few 

PEDBOR 

0.01217 

0.00586 

5.22317 

No 

No 

5 

MedSob  (W=3,  T=30  ) 

0.02987 

0.02165 

14.5882 

No 

No 

SIGMR  (T1=50,T2=15) 

0.24336 

0.00198 

23.4948 

Few 

Many 

PEDBOR 

0.02655 

0.01438 

11.2249 

No 

No 

3 

MedSob  (1)  (W=5,  T=20) 

0.12058 

0.06107 

37.1881 

Some 

Few 

MedSob  (2)  (W=5,  T=30) 

0.43363 

0.00967 

37.1881 

No 

Many 

SIGMR  (Tl=50,  T2=12) 

0.27544 

0.04430 

50.3117 

Some 

Many 

PEDBOR 

0.10398 

0.03564 

32.2599 

Few 

No 

1 

MedSob  (1)  (W=5,  T=20) 

0.22235 

0.20876 

91.1897 

Many 

Some 

MedSob  (2)  (W=7,  T=20) 

0.89718 

0.03817 

63.7568 

Some 

Many 

SIGMR(T1=50,  T2=15) 

0.27765 

0.31591 
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Fig.  9-1  4-level  Images 
(a)  Original  image,  (b)  7 dB  noisy  image. 

(c)  5 dB  noisy  image,  (d)  3 dB  noisy  image,  (e)  1 dB  noisy  image 
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Fig.  9-2  Bar  Images 

(a)  Original  image,  (b)  7 dB  noisy  image. 

(c)  5 dB  noisy  image,  (d)  3 dB  noisy  image,  (e)  1 dB  noisy  image. 
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Fig.  9-3  Checkerboard  Images 
(a)  Original  image,  (b)  7 dB  noisy  image. 

(c)  5 dB  noisy  image,  (d)  3 dB  noisy  image,  (e)  1 dB  noisy  image. 
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Fig.  9-4  Edge  Detection  Results  for  the  4-level  Images 
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(a)  Original  Edges. 


(b)  The  Results  of  the  PEDBOR. 
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(c)  The  Results  of  the  MedSob  (1). 


Fig.  9-5  Edge  Detection  Results  for  Bar  Images 
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Fig.  9-6  Edge  Detection  Results  for  Checkerboard  Images 
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(a)  Original  Image 


(b)  The  Result  of  the  PEDBOR 
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(d)  The  Results  of  the  SIGMR. 


Fig.  9-7  Noise  Removed  Images  for  4-level  Images 
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(a)  Original  Image 
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(b)  The  Results  of  the  PEDBOR. 
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Fig.  9-8  Noise  Removed  Images  for  Bar  Images. 
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(c)  The  Results  of  the  MedSob. 
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Fig.  9-9  Noise  Removed  Images  for  Checkerboard  Images. 
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Fig.  9-10  Evaluation  of  Image  Surface 
(a)  Original  image  surface,  (b)  Noise  image  surface  (SNR=3db). 
(c)  PEDBOR  surface,  (d)  SIGMR  surface,  (e)  MedSob  surface. 
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(a)  Original  Image 


(b)  Noisy  Image  (a  = 20 ). 


(c)  Noisy  Image  (a  = 30  ). 


Fig.  9-11  Lena  Image  and  Corresponding  Noisy  Images. 
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(a)  Original  Edges. 


(b)  PEDBOR  Result  for  a = 20. 


(c)  PEDBOR  Result  for  a = 30. 


Fig.  9-12  Edge  Detection  Results  for  Lena  Images. 
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(d)  MedSob  Results  for  a = 20. 


Fig.  9-12  Continued 
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(e)  MedSob  Results  for  o = 30. 
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(f)  SIGMR  Results  for  a = 20. 


Fig.  9-12  Continued 
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Tl=50,  T2=10 


(g)  SIGMR  Results  for  o = 30. 


(h)  LoG  Results  for  a = 20. 


Fig.  9-12  Continued 
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(a)  Original  Image 


(b)  Noisy  Image  (a  = 20 ) 


(c)  PEDBOR  Result. 


Fig.  9-13  Noise  Removed  Results  for  Lena's  Noisy  Image  with  a = 20. 
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W = 3 
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(d)  The  Results  of  MedSob  with  the  Median  Filters  of  Size  W. 


Fig.  9-13  Continued 
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Tl=20  Tl=50 


(e)  The  Results  of  SIGMR  with  variation  of  the  Regularization  Parameter  T 1 . 


Fig.  9-13  Continued 
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Fig.  9-14  Experimental  Results  for  Malignant  Melanoma  Image  1. 
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Fig.  9-15  Experimental  Results  for  Malignant  Melanoma  Image  2. 


132 


Fig.  9-16  Experimental  Results  for  Malignant  Melanoma  Image  3. 


1 1. 
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Fig.  9-17  Experimental  Results  for  Malignant  Melanoma  Image  4. 
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Fig.  9-18  Experimental  Results  for  Malignant  Melanoma  Image  5. 
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Fig.  9-18  Experimental  Results  for  Malignant  Melanoma  Image  6. 


CHAPTER  10 

SUMMARY  AND  CONCLUSIONS 


10.1  Summary 

The  objective  of  this  study  is  to  develop  a new  segmentation  approach  for  a 
computer  diagnostic  system  of  malignant  melanoma  and  to  show  that  the  developed 
approach  outperforms  traditional  approaches. 

The  developed  approach  is  an  adaptive,  optimal  and  integrating  one.  It  has  three 
stages:  1)  PEDBOR;  2)  LVDSO  quantization;  3)  mapping.  In  Stage  I,  a new  algorithm, 
PEDBOR,  is  used  to  remove  noise  and  detect  edges  simultaneously.  It  is  based  on  the 
regularization  model,  but  has  critical  improvements.  With  the  improvements,  the  whole 
regularization  procedure  is  adaptively  controlled  by  pixel  features,  which  makes  the 
algorithm  robust  to  noise.  In  Stage  II,  an  iterative  LVDSO  quantization  is  applied  to 
quantize  noise  removed  images  into  initially  segmented  images.  Stage  III  is  a mapping 
procedure.  It  uses  edge  images  and  initially  segmented  images  as  references  to  remove 
false  edges  and  false  regions.  The  experimental  results  show  that  the  approach  can 
generate  uniform,  compact  and  dissimilar  segmentation  results.  The  segmentation  results 
are  also  consistent  with  dermatologists'  observation  results. 

10.2  Significance  of  This  Research 

A new  and  promising  hybrid  segmentation  approach  for  a computer  diagnostic 
system  of  malignant  melanoma  is  proposed  in  this  study.  For  a practical  computer  image 
analysis  systems,  such  as  the  computer  diagnostic  system  of  malignant  melanoma 
described  here,  there  are  many  kinds  of  image  sources,  which  require  that  the  algorithm 
must  be  robust  to  various  kinds  of  noise  and  that  image  processing  parameters  change 
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over  a wide  range.  Also  such  systems  will  be  used  by  many  who  lack  image  processing 
knowledge,  which  makes  choosing  processing  parameters  difficult  for  them.  Therefore, 
the  segmentation  algorithm  for  the  system  must  be  robust  to  noise  and  be  universally 
suitable  to  the  class  of  application  images  without  the  difficulty  for  user  input  of 
processing  parameters.  The  developed  hybrid  segmentation  approach  satisfies  these 
requirements. 

The  developed  approach  is  superior  to  traditional  approaches  in  several  aspects. 

(1)  The  approach  avoids  the  difficulty  with  setting  a threshold  in  edge  detection  and 
setting  clustering  parameters  in  quantization.  The  results  of  our  approach  are 
minimally  dependent  on  the  threshold/parameter  setting. 

(2)  PEDBOR  uses  a pixel-feature-controlling  regularization  process  to  avoid 
oversmoothing  of  edges  and  undersuppression  of  noise.  This  makes  the  approach 
much  more  robust  to  various  sources  of  noise  than  other  traditional  approaches. 

(3)  Combining  edge  based  segmentation  results  with  region  based  segmentation  results 
makes  segmentation  results  more  consistent. 

By  using  this  approach,  the  segmented  results  better  define  the  image  features. 
This  provides  a consistent  model  for  high  level  analysis,  such  as  texture,  color,  shape,  size, 
etc.,  and  for  classification  of  a malignant  melanoma  lesion. 

10.2  Recommendations  for  Future  Research 
Since  the  proposed  approach  is  new,  many  follow-up  studies  are  possible. 
Specifically,  the  studies  in  speeding  up  the  algorithms  are  of  high  interest  in  the  fiiture. 
Several  aspects  in  the  speed  up  of  the  algorithms  can  be  considered.  First,  reduce  the 
computation  redundancy  in  implementing  the  algorithms.  Second,  consider  the  strategy  of 
the  parallel  implementation  of  the  algorithms.  Third,  implement  the  algorithms  in 
hardware.  How  the  algorithms  can  be  efiBciently  implemented  in  parallel  and  in  hardware 
remains  to  be  investigated. 
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577 

215 

217 

-90 

0 

0 

114 

115 

-48 

0 

0 

578 

218 

219 

-91 

0 

0 

116 

118 

-49 

0 

0 

579 

220 

222 

-92 

0 

0 

119 

120 

-50 

0 

0 

580 

223 

224 

-93 

0 

0 

121 

123 

-51 

0 

0 

581 

225 

226 

-94 

0 

0 

124 

125 

-52 

0 

0 

582 

227 

229 

-95 

0 

0 

126 

127 

-53 

0 

0 

583 

230 

231 

-96 

0 

0 

128 

130 

-54 

0 

0 

584 

232 

234 

-97 

0 

0 

Index 

585 

586 

587 

588 

589 

590 

591 

592 

593 

594 

595 

596 

597 

598 

599 

600 

601 

602 

603 

604 

605 

606 

607 

608 

609 

610 

611 

612 

613 

614 

615 

616 

617 

618 

619 

620 

621 

622 

623 

624 

625 

626 
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Ymax  Ymin  Xmax  Xmin  OG  Index  Ymax  Ymin  Xmax  Xmin  OG 


235 

236 

-98 

0 

0 

627 

-176 

-174 

72 

0 

32 

237 

239 

-99 

0 

0 

628 

-173 

-172 

71 

0 

32 

240 

241 

-100 

0 

0 

629 

-171 

-169 

70 

0 

32 

242 

243 

-101 

0 

0 

630 

-168 

-167 

69 

0 

32 

244 

246 

-102 

0 

0 

631 

-166 

-165 

68 

0 

32 

247 

248 

-103 

0 

0 

632 

-164 

-162 

67 

0 

32 

249 

251 

-104 

0 

0 

633 

-161 

-160 

66 

0 

32 

252 

253 

-105 

0 

0 

634 

-159 

-157 

65 

0 

32 

254 

-256 

106 

0 

32 

635 

-156 

-155 

64 

0 

32 

-255 

-254 

105 

0 

32 

636 

-154 

-153 

63 

0 

32 

-253 

-252 

104 

0 

32 

637 

-152 

-150 

62 

0 

32 

-251 

-249 

103 

0 

32 

638 

-149 

-148 

61 

0 

32 

-248 

-247 

102 

0 

32 

639 

-147 

-145 

60 

0 

32 

-246 

-244 

101 

0 

32 

640 

-144 

-143 

59 

0 

32 

-243 

-242 

100 

0 

32 

641 

-142 

-141 

58 

0 

32 

-241 

-240 

99 

0 

32 

642 

-140 

-138 

57 

0 

32 

-239 

-237 

98 

0 

32 

643 

-137 

-136 

56 

0 

32 

-236 

-235 

97 

0 

32 

644 

-135 

-133 

55 

0 

32 

-234 

-232 

96 

0 

32 

645 

-132 

-131 

54 

0 

32 

-231 

-230 

95 

0 

32 

646 

-130 

-128 

53 

0 

32 

-229 

-227 

94 

0 

32 

647 

-127 

-126 

52 

0 

32 

-226 

-225 

93 

0 

32 

648 

-125 

-124 

51 

0 

32 

-224 

-223 

92 

0 

32 

649 

-123 

-121 

50 

0 

32 

-222 

-220 

91 

0 

32 

650 

-120 

-119 

49 

0 

32 

-219 

-218 

90 

0 

32 

651 

-118 

-116 

48 

0 

32 

-217 

-215 

89 

0 

32 

652 

-115 

-114 

47 

0 

32 

-214 

-213 

88 

0 

32 

653 

-113 

-112 

46 

0 

32 

-212 

-211 

87 

0 

32 

654 

-111 

-109 

45 

0 

32 

-210 

-208 

86 

0 

32 

655 

-108 

-107 

44 

0 

32 

-207 

-206 

85 

0 

32 

656 

-106 

-106 

44 

254 

32 

-205 

-203 

84 

0 

32 

657 

-105 

-105 

44 

252 

32 

-202 

-201 

83 

0 

32 

658 

-104 

-104 

43 

249 

32 

-200 

-198 

82 

0 

32 

659 

-103 

-103 

43 

247 

32 

-197 

-196 

81 

0 

32 

660 

-102 

-102 

42 

244 

32 

-195 

-194 

80 

0 

32 

661 

-101 

-101 

42 

242 

32 

-193 

-191 

79 

0 

32 

662 

-100 

-100 

42 

240 

32 

-190 

-189 

78 

0 

32 

663 

-99 

-99 

41 

237 

32 

-188 

-186 

77 

0 

32 

664 

-98 

-98 

41 

235 

32 

-185 

-184 

76 

0 

32 

665 

-97 

-97 

40 

232 

32 

-183 

-182 

75 

0 

32 

666 

-96 

-96 

40 

230 

32 

-181 

-179 

74 

0 

32 

667 

-95 

-95 

39 

227 

32 

-178 

-177 

73 

0 

32 

668 

-94 

-94 

39 

225 

32 

669 

-93 

-93 

39 

223 

32 

Index 

670 

671 

672 

673 

674 

675 

676 

677 

678 

679 

680 

681 

682 

683 

684 

685 

686 

687 

688 

689 

690 

691 

692 

693 

694 

695 

696 

697 

698 

699 

700 

701 

702 

703 

704 

705 

706 

707 

708 

709 

710 

711 

712 
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Ymax  Ymin  Xmax  Xmin  OG  Index  Ymax  Ymin  Xmax  Xmin  OG 


-92 

-92 

38 

220 

32 

713 

-49 

-49 

20 

116 

32 

-91 

-91 

38 

218 

32 

714 

-48 

-48 

20 

114 

32 

-90 

-90 

37 

215 

32 

715 

-47 

-47 

20 

112 

32 

-89 

-89 

37 

213 

32 

716 

-46 

-46 

19 

109 

32 

-88 

00 

00 

1 

37 

211 

32 

717 

-45 

-45 

19 

107 

32 

-87 

-87 

36 

208 

32 

718 

-44 

-44 

18 

104 

32 

-86 

-86 

36 

206 

32 

719 

-43 

-43 

18 

102 

32 

-85 

-85 

35 

203 

32 

720 

-42 

-42 

17 

99 

32 

-84 

-84 

35 

201 

32 

721 

-41 

-41 

17 

97 

32 

-83 

-83 

34 

198 

32 

722 

-40 

-40 

17 

95 

32 

-82 

-82 

34 

196 

32 

723 

-39 

-39 

16 

92 

32 

-81 

-81 

34 

194 

32 

724 

-38 

-38 

16 

90 

32 

-80 

-80 

33 

191 

32 

725 

-37 

-37 

15 

87 

32 

-79 

-79 

33 

189 

32 

726 

-36 

-36 

15 

85 

32 

-78 

-78 

32 

186 

32 

727 

-35 

-35 

15 

83 

32 

-77 

-77 

32 

184 

32 

728 

-34 

-34 

14 

80 

32 

-76 

-76 

32 

182 

32 

729 

-33 

-33 

14 

78 

32 

-75 

-75 

31 

179 

32 

730 

-32 

-32 

13 

75 

32 

-74 

-74 

31 

177 

32 

731 

-31 

-31 

13 

73 

32 

-73 

-73 

30 

174 

32 

732 

-30 

-30 

13 

71 

32 

-72 

-72 

30 

172 

32 

733 

-29 

-29 

12 

68 

32 

-71 

-71 

29 

169 

32 

734 

-28 

-28 

12 

66 

32 

-70 

-70 

29 

167 

32 

735 

-27 

-27 

11 

63 

32 

-69 

-69 

29 

165 

32 

736 

-26 

-26 

11 

61 

32 

-68 

-68 

28 

162 

32 

737 

-25 

-25 

10 

58 

32 

-67 

-67 

28 

160 

32 

738 

-24 

-24 

10 

56 

32 

-66 

-66 

27 

157 

32 

739 

-23 

-23 

10 

54 

32 

-65 

-65 

27 

155 

32 

740 

-22 

-22 

9 

51 

32 

-64 

-64 

27 

153 

32 

741 

-21 

-21 

9 

49 

32 

-63 

-63 

26 

150 

32 

742 

-20 

-20 

8 

46 

32 

-62 

-62 

26 

148 

32 

743 

-19 

-19 

8 

44 

32 

-61 

-61 

25 

145 

32 

744 

-18 

-18 

8 

42 

32 

-60 

-60 

25 

143 

32 

745 

-17 

-17 

7 

39 

32 

-59 

-59 

25 

141 

32 

746 

-16 

-16 

7 

37 

32 

-58 

-58 

24 

138 

32 

747 

-15 

-15 

6 

34 

32 

-57 

-57 

24 

136 

32 

748 

-14 

-14 

6 

32 

32 

-56 

-56 

23 

133 

32 

749 

-13 

-13 

5 

29 

32 

-55 

-55 

23 

131 

32 

750 

-12 

-12 

5 

27 

32 

-54 

-54 

22 

128 

32 

751 

-11 

-11 

5 

25 

32 

-53 

-53 

22 

126 

32 

752 

-10 

-10 

4 

22 

32 

-52 

-52 

22 

124 

32 

753 

-9 

-9 

4 

20 

32 

-51 

-51 

21 

121 

32 

754 

-8 

-8 

3 

17 

32 

o 

1 

-50 

21 

119 

32 

755 

-7 

-7 

3 

15 

32 

Index 

756 

757 

758 

759 

760 

761 

762 

763 

764 

765 

766 

767 

768 

769 

770 

771 

772 

773 

774 

775 

776 

777 

778 

779 

780 

781 

782 

783 

784 

785 

786 

787 

788 

789 

790 

791 

792 

793 

794 

795 

796 

797 

798 
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Ymax 

Ymin 

Xmax  Xmin 

i OG 

Index 

Ymax 

Ymin 

Xmax  Xmin 

OG 

-6 

-6 

3 

13 

32 

799 

-70 

-70 

-255 

-166 

192 

-5 

-5 

2 

10 

32 

800 

-69 

-69 

-255 

-164 

192 

-4 

-4 

2 

8 

32 

1801 

-68 

-68 

-255 

-161 

192 

-3 

-3 

1 

5 

32 

802 

-67 

-67 

-255 

-159 

192 

-2 

-2 

1 

3 

32 

803 

-66 

-66 

-255 

-156 

192 

-1 

-1 

-255 

3 

32 

804 

-65 

-65 

-255 

-154 

192 

0 

-256 

-255 

3 

192 

805 

-64 

-64 

-255 

-152 

192 

-255 

-106 

-255 

-253 

192 

806 

-63 

-63 

-255 

-149 

192 

-105 

-105 

-255 

-251 

192 

807 

-62 

-62 

-255 

-147 

192 

-104 

-104 

-255 

-248 

192 

808 

-61 

-61 

-255 

-144 

192 

-103 

-103 

-255 

-246 

192 

809 

-60 

-60 

-255 

-142 

192 

-102 

-102 

-255 

-243 

192 

810 

-59 

-59 

-255 

-140 

192 

-101 

-101 

-255 

-241 

192 

811 

-58 

-58 

-255 

-137 

192 

-100 

-100 

-255 

-239 

192 

812 

-57 

-57 

-255 

-135 

192 

-99 

-99 

-255 

-236 

192 

813 

-56 

-56 

-255 

-132 

192 

-98 

-98 

-255 

-234 

192 

814 

-55 

-55 

-255 

-130 

192 

-97 

-97 

-255 

-231 

192 

815 

-54 

-54 

-255 

-127 

192 

-96 

-96 

-255 

-229 

192 

816 

-53 

-53 

-255 

-125 

192 

-95 

-95 

-255 

-226 

192 

817 

-52 

-52 

-255 

-123 

192 

-94 

-94 

-255 

-224 

192 

818 

-51 

-51 

-255 

-120 

192 

-93 

-93 

-255 

-222 

192 

819 

-50 

-50 

-255 

-118 

192 

-92 

-92 

-255 

-219 

192 

820 

-49 

-49 

-255 

-115 

192 

-91 

-91 

-255 

-217 

192 

821 

-48 

-48 

-255 

-113 

192 

-90 

-90 

-255 

-214 

192 

822 

-47 

-47 

-255 

-111 

192 

-89 

-89 

-255 

-212 

192 

823 

-46 

-46 

-255 

-108 

192 

-88 

-88 

-255 

-210 

192 

824 

-45 

-45 

-255 

-106 

192 

-87 

-87 

-255 

-207 

192 

825 

-44 

-44 

-255 

-103 

192 

-86 

-86 

-255 

-205 

192 

826 

-43 

-43 

-255 

-101 

192 

-85 

-85 

-255 

-202 

192 

827 

-42 

-42 

-255 

-98 

192 

-84 

-84 

-255 

-200 

192 

828 

-41 

-41 

-255 

-96 

192 

-83 

-83 

-255 

-197 

192 

829 

-40 

-40 

-255 

-94 

192 

-82 

-82 

-255 

-195 

192 

830 

-39 

-39 

-255 

-91 

192 

-81 

-81 

-255 

-193 

192 

831 

-38 

-38 

-255 

-89 

192 

-80 

-80 

-255 

-190 

192 

832 

-37 

-37 

-255 

-86 

192 

-79 

-79 

-255 

-188 

192 

833 

-36 

-36 

-255 

-84 

192 

-78 

-78 

-255 

-185 

192 

834 

-35 

-35 

-255 

-82 

192 

-77 

-77 

-255 

-183 

192 

835 

-34 

-34 

-255 

-79 

192 

-76 

-76 

-255 

-181 

192 

836 

-33 

-33 

-255 

-77 

192 

-75 

-75 

-255 

-178 

192 

837 

-32 

-32 

-255 

-74 

192 

-74 

-74 

-255 

-176 

192 

838 

-31 

-31 

-255 

-72 

192 

-73 

-73 

-255 

-173 

192 

839 

-30 

-30 

-255 

-70 

192 

-72 

-72 

-255 

-171 

192 

840 

-29 

-29 

-255 

-67 

192 

-71 

-71 

-255 

-168 

192 

841 

-28 

-28 

-255 

-65 

192 

Index 

842 

843 

844 

845 

846 

847 

848 

849 

850 

851 

852 

853 

854 

855 

856 

857 

858 

859 

860 

861 

862 

863 

864 

865 

866 

867 

868 

869 

870 

871 

872 

873 

874 

875 

876 

877 

878 

879 

880 

881 

882 

883 

884 
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Ymax  Ymin  Xmax  Xmin  OG  Index  Ymax  Ymin  Xmax  Xmin  OG 


-27 

-27 

-255 

-62 

192 

885 

16 

16 

-255 

-41 

192 

-26 

-26 

-255 

-60 

192 

886 

17 

17 

-255 

-43 

192 

-25 

-25 

-255 

-57 

192 

887 

18 

18 

-255 

-45 

192 

-24 

-24 

-255 

-55 

192 

888 

19 

19 

-255 

-48 

192 

-23 

-23 

-255 

-53 

192 

889 

20 

20 

-255 

-50 

192 

-22 

-22 

-255 

-50 

192 

890 

21 

21 

-255 

-53 

192 

-21 

-21 

-255 

-48 

192 

891 

22 

22 

-255 

-55 

192 

-20 

-20 

-255 

-45 

192 

892 

23 

23 

-255 

-57 

192 

-19 

-19 

-255 

-43 

192 

893 

24 

24 

-255 

-60 

192 

-18 

-18 

-255 

-41 

192 

894 

25 

25 

-255 

-62 

192 

-17 

-17 

-255 

-38 

192 

895 

26 

26 

-255 

-65 

192 

-16 

-16 

-255 

-36 

192 

896 

27 

27 

-255 

-67 

192 

-15 

-15 

-255 

-33 

192 

897 

28 

28 

-255 

-70 

192 

-14 

-14 

-255 

-31 

192 

898 

29 

29 

-255 

-72 

192 

-13 

-13 

-255 

-28 

192 

899 

30 

30 

-255 

-74 

192 

-12 

-12 

-255 

-26 

192 

900 

31 

31 

-255 

-77 

192 

-11 

-11 

-255 

-24 

192 

901 

32 

32 

-255 

-79 

192 

-10 

-10 

-255 

-21 

192 

902 

33 

33 

-255 

-82 

192 

-9 

-9 

-255 

-19 

192 

903 

34 

34 

-255 

-84 

192 

-8 

-8 

-255 

-16 

192 

904 

35 

35 

-255 

-86 

192 

-7 

-7 

-255 

-14 

192 

905 

36 

36 

-255 

-89 

192 

-6 

-6 

-255 

-12 

192 

906 

37 

37 

-255 

-91 

192 

-5 

-5 

-255 

-9 

192 

907 

38 

38 

-255 

-94 

192 

-4 

-4 

-255 

-7 

192 

908 

39 

39 

-255 

-96 

192 

-3 

-3 

-255 

-4 

192 

909 

40 

40 

-255 

-98 

192 

-2 

-2 

-255 

-2 

192 

910 

41 

41 

-255 

-101 

192 

-1 

-1 

-255 

0 

192 

911 

42 

42 

-255 

-103 

192 

0 

0 

-255 

-2 

192 

912 

43 

43 

-255 

-106 

192 

1 

1 

-255 

-4 

192 

913 

44 

44 

-255 

-108 

192 

2 

2 

-255 

-7 

192 

914 

45 

45 

-255 

-111 

192 

3 

3 

-255 

-9 

192 

915 

46 

46 

-255 

-113 

192 

4 

4 

-255 

-12 

192 

916 

47 

47 

-255 

-115 

192 

5 

5 

-255 

-14 

192 

917 

48 

48 

-255 

-118 

192 

6 

6 

-255 

-16 

192 

918 

49 

49 

-255 

-120 

192 

7 

7 

-255 

-19 

192 

919 

50 

50 

-255 

-123 

192 

8 

8 

-255 

-21 

192 

920 

51 

51 

-255 

-125 

192 

9 

9 

-255 

-24 

192 

921 

52 

52 

-255 

-127 

192 

10 

10 

-255 

-26 

192 

922 

53 

53 

-255 

-130 

192 

11 

11 

-255 

-28 

192 

923 

54 

54 

-255 

-132 

192 

12 

12 

-255 

-31 

192 

924 

55 

55 

-255 

-135 

192 

13 

13 

-255 

-33 

192 

925 

56 

56 

-255 

-137 

192 

14 

14 

-255 

-36 

192 

926 

57 

57 

-255 

-140 

192 

15 

15 

-255 

-38 

192 

927 

58 

58 

-255 

-142 

192 

Index 

928 

929 

930 

931 

932 

933 

934 

935 

936 

937 

938 

939 

940 

941 

942 

943 

944 

945 

946 

947 

948 

949 

950 

951 

952 

953 

954 

955 

956 

957 

958 

959 

960 

961 

962 

963 

964 

965 

966 

967 

968 

969 

970 
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Ymax  Ymin 

Xmax  Xmin 

OG 

Index 

Ymax 

Ymin 

1 Xmax  Xmin 

OG 

59 

59 

-255 

-144 

192 

971 

102 

102 

-255 

-248 

192 

60 

60 

-255 

-147 

192 

972 

103 

103 

-255 

-251 

192 

61 

61 

-255 

-149 

192 

973 

104 

104 

-255 

-253 

192 

62 

62 

-255 

-152 

192 

974 

105 

-256 

-255 

-253 

64 

63 

63 
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1372 
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253 
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-105 

224 
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41 

41 

18 

102 

96 

254 

-256 

-255 
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96 

1402 

42 
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-255 

0 

1 

3 

96 

1403 

43 

43 

19 

107 

96 

1 

1 

1 

5 

96 

1404 

44 

44 

19 

109 

96 

2 

2 

2 

8 

96 

1405 

45 

45 

20 

112 

96 

3 

3 

2 

10 

96 

1406 

46 

46 

20 

114 

96 

4 

4 

3 

13 

96 

1407 

47 

47 

20 

116 

96 

5 

5 

3 

15 

96 

1408 

48 

48 

21 

119 

96 

6 

6 

3 

17 

96 

1409 

49 

49 

21 

121 

96 

7 

7 

4 

20 

96 

1410 

50 

50 

22 

124 

96 

8 

8 

4 

22 

96 

1411 

51 

51 

22 

126 

96 

9 

9 

5 

25 

96 

1412 

52 

52 

22 

128 

96 

10 

10 

5 

27 

96 

1413 

53 

53 

23 

131 

96 

11 

11 

5 

29 

96 

1414 

54 

54 

23 

133 

96 

12 

12 

6 

32 

96 

1415 

55 

55 

24 

136 

96 

13 

13 

6 

34 

96 

1416 

56 

56 

24 

138 
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14 

14 

7 

37 

96 

1417 

57 

57 

25 

141 

96 

15 

15 

7 

39 

96 

1418 

58 

58 

25 

143 

96 

16 

16 

8 

42 

96 

1419 

59 

59 

25 

145 

96 

17 

17 

8 

44 

96 

1420 

60 

60 

26 

148 

96 

18 

18 

8 

46 

96 

1421 

61 

61 

26 

150 

96 

19 

19 

9 

49 

96 

1422 

62 

62 

27 

153 

96 

20 

20 

9 

51 

96 

1423 

63 

63 

27 

155 

96 

21 

21 

10 

54 

96 

1424 

64 

64 

27 

157 

96 

22 

22 

10 

56 

96 
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65 

65 

28 

160 

96 

23 

23 

10 

58 

96 
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66 

66 

28 

162 
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24 

24 

11 

61 

96 
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67 

67 

29 

165 

96 

25 

25 

11 

63 

96 
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68 

68 

29 

167 

96 

26 

26 

12 

66 

96 
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69 

69 

29 

169 

96 

27 

27 

12 

68 

96 

1430 

70 

70 

30 

172 

96 

28 

28 

13 

71 

96 

1431 

71 

71 

30 

174 

96 

29 

29 

13 

73 

96 

1432 

72 

72 

31 

177 

96 

30 

30 

13 

75 

96 

1433 

73 

73 

31 

179 

96 

31 

31 

14 

78 

96 

1434 

74 

74 

32 

182 

96 

32 

32 

14 

80 

96 

1435 

75 

75 

32 

184 

96 

33 

33 

15 

83 

96 

1436 

76 

76 

32 

186 

96 

34 

34 

15 

85 

96 

1437 

77 

77 

33 

189 

96 

35 

35 

15 

87 

96 

1438 

78 

78 

33 

191 

96 

36 

36 

16 

90 

96 

1439 

79 

79 

34 

194 

96 

37 

37 

16 

92 

96 

1440 

80 

80 

34 

196 

96 

38 

38 

17 

95 

96 

1441 

81 

81 

34 

198 

96 

39 

39 

17 

97 

96 

1442 

82 

82 

35 

201 

96 

40 

40 

17 

99 

96 

1443 

83 

83 

35 

203 

96 

Index 

1444 
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1448 
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1450 
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1453 
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1455 
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1457 

1458 

1459 

1460 
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1462 

1463 

1464 

1465 

1466 

1467 

1468 

1469 

1470 
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1472 
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84 

84 

36 

206 

96 

1486 

155 

156 

66 

254 

96 

85 

85 

36 

208 

96 

1487 

157 

159 

67 
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96 

86 

86 

37 

211 

96 

1488 

160 

161 

68 
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96 

87 

87 

37 

213 

96 

1489 

162 

164 

69 
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96 

88 

88 

37 

215 

96 
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165 
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70 
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96 

89 

89 

38 

218 

96 

1491 
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168 

71 
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96 

90 

90 

38 

220 

96 

1492 
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171 

72 
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96 

91 

91 

39 

223 

96 

1493 

172 

173 

73 
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96 

92 

92 

39 

225 

96 

1494 

174 

176 

74 
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96 

93 

93 

39 

227 

96 

1495 

177 

178 

75 
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96 

94 

94 

40 

230 

96 

1496 

179 

181 

76 
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96 

95 

95 

40 

232 

96 

1497 

182 

183 

77 
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96 

96 

96 

41 

235 

96 

1498 

184 

185 

78 
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96 

97 

97 

41 

237 

96 

1499 

186 
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79 
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96 

98 

98 

42 

240 

96 
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80 
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96 

99 

99 

42 

242 

96 

1501 
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193 

81 
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96 
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42 

244 

96 
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195 

82 
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96 
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43 

247 
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1503 

196 

197 

83 
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96 
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43 

249 
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198 

200 

84 
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44 
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89 

254 
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49 
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91 
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96 

116 
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219 

92 
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51 
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52 
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96 
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254 

96 
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53 
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55 
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96 

1518 
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96 
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96 
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96 

138 
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59 
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60 
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96 
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243 
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61 
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96 

1523 
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96 
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62 
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96 
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104 
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96 

148 
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63 
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96 
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105 
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96 
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64 
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96 
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106 

254 

96 

153 
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65 

254 

96 
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